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ABSTRACT 


The  history  of  man  is  written  in  choice.  Whether  simple  or  complex,  on  a  whim 
or  after  labored  consideration,  inflamed  by  passion  or  calculated  coolly,  the  judgments 
that  we  form  and  the  choices  that  we  make  define  who  we  are  and  what  we  want  for  the 
future.  Yet  most  of  us  have  little  or  no  conscious  awareness  of  the  inner  workings  of  our 
own  minds.  We  often  choose  without  understanding  or  accounting  for  the  perceptions, 
intuitions,  and  inferences  that  underlie  our  decisions.  So  how  do  people  make  decisions? 
How  do  we  cope  with  the  volume  and  complexity  of  information  in  our  environment 
without  being  overwhelmed?  How  do  we  use  our  senses  to  select  and  process  this 
information,  and  how  do  we  organize,  contextualize,  and  conceptualize  it  once  it  reaches 
our  brains?  How  do  we  form  judgments  about  the  value  of  a  specific  piece  of 
information  or  about  the  likelihood  of  a  particular  event  or  outcome?  And  what  are  the 
factors  that  lead  us  astray?  The  search  for  answers  to  these  questions  is  more  than 
academic;  understanding  the  fundamentals  of  perception  and  cognition  is  critical  to 
effective  analysis  and  decision-making.  For  those  involved  in  national  security,  and 
particularly  for  those  involved  in  the  collection  and  analysis  of  national  intelligence,  an 
appreciation  of  the  intricacies  of  these  processes  has  real-world  implications.  As 
evidenced  by  the  dramatic  intelligence  failures  of  the  last  few  years,  and  in  particular  by 
the  mistaken  assessment  concerning  Iraqi  weapons  of  mass  destruction,  understanding 
how  we  arrive  at  judgments  and  decisions  can  be  quite  literally  a  matter  of  life  and  death. 
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I.  INTRODUCTION 


The  history  of  man  is  written  in  choice.  Whether  simple  or  complex,  on  a  whim 
or  after  labored  consideration,  inflamed  by  passion  or  calculated  coolly,  the  judgments 
we  form  and  the  choices  we  make  define  who  we  are  and  what  we  want  for  the  future. 

Yet  most  of  us  have  little  or  no  conscious  awareness  of  the  inner  workings  of  our 
own  minds.  The  functions  associated  with  choice  in  particular  are  often  “conducted  prior 
to  and  independent  of  any  conscious  direction”  (Heuer,  1999,  p.  3).  What  spontaneously 
appears  in  our  consciousness  at  the  end  of  cognition  is  the  product,  not  the  process,  of 
thinking.  We  often  choose  without  understanding  or  accounting  for  the  perceptions, 
intuitions,  and  inferences  that  underlie  our  decisions. 

So  how  do  people  make  decisions?  How  do  we  cope  with  the  volume  and 
complexity  of  information  in  our  environment  without  being  overwhelmed?  How  do  we 
use  our  senses  to  select  and  process  this  information,  and  how  do  we  organize, 
contextualize,  and  conceptualize  it  once  it  reaches  our  brains?  How  do  we  form 
judgments  about  the  value  of  a  specific  piece  of  infonnation  or  about  the  likelihood  of  a 
particular  event?  And  what  are  the  factors  that  lead  us  astray? 

The  search  for  answers  to  these  questions  is  more  than  academic;  understanding 
the  fundamentals  of  perception  and  cognition  is  critical  to  effective  analysis  and  decision¬ 
making.  For  those  involved  in  national  security,  and  particularly  for  those  charged  with 
the  collection  and  analysis  of  national  intelligence,  an  appreciation  of  the  intricacies  of 
these  processes  has  real-world  implications.  As  evidenced  by  the  dramatic  intelligence 
failures  of  the  last  few  years,  understanding  how  we  arrive  at  judgments  and  decisions 
can  be  quite  literally  a  matter  of  life  and  death. 

A.  THEORIES  OF  CHOICE 

1.  The  Classic  Theory 

Any  discussion  of  choice  must  begin  with  an  acknowledgment  of  the  long  shadow 
cast  by  the  classical  model  of  rational  choice.  Applied  most  vigorously  in  the  study  of 
economics,  the  model’s  influence  is  apparent  in  virtually  all  of  the  behavioral  and  social 
sciences.  According  to  this  model,  people  choose  which  options  to  pursue  by  assessing 
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the  probability  of  each  possible  outcome,  discerning  the  utility  to  be  derived  from  each, 
and  combining  the  two  assessments.  We  pursue  the  option  that  offers  the  optimal 
combination  of  probability  and  utility  (Gilovich,  Griffin,  &  Kahneman,  2002,  p.  l)d 

“Calculations  of  probability  and  multi-attribute  utility  can  be  rather  formidable 
judgments  to  make,  yet  the  theory  of  rational  choice  assumes  that  people  make  them,  and 
make  them  well”  (Gilovich  et  al,  2002,  p.  2).  Proponents  of  this  theory  do  not  believe 
that  people  are  flawless  in  these  calculations,  but  they  do  insist  that  any  mistakes  are  not 
systematic.  The  model  assumes  that  a  rational  actor  will  follow  the  elementary  rules  of 
probability  and  expected  value  when  calculating,  for  example,  the  likelihood  of  a 
particular  candidate  winning  an  election,  the  risks  involved  in  a  surgical  procedure,  or  the 
odds  of  winning  a  game  of  poker  (p.  2). 

The  rational  choice  model  has  a  number  of  important  implications.  If  we  accept 
an  actor’s  values  as  given  (and  consistent),  if  we  postulate  an  objective  description  of  the 
world  as  it  really  is,  and  if  we  assume  the  actor’s  computational  powers  are  unlimited, 
then  two  important  consequences  follow.  “First,  we  do  not  need  to  distinguish  between 
the  real  world  and  the  rational  actor’s  perception  of  it;  he  or  she  perceives  the  world  as  it 
really  is.  Second,  we  can  predict  the  choices  that  will  be  made  by  a  rational  actor  entirely 
from  our  knowledge  of  the  real  world  and  without  knowledge  of  the  actor’s  perceptions 
or  modes  of  calculation”  (Simon,  1986,  pp.  210-211). 

Yet  is  the  average  person  as  attuned  to  the  axioms  of  formal  rationality  as  this 
stance  demands?  Countless  studies  and  experiments  over  the  past  half  century  indicate 
that  people’s  assessments  of  likelihood  and  risk  do  not  conform  to  the  mathematical  laws 
necessary  to  validate  this  theory.  In  one  of  the  earliest  empirical  studies  on  the  subject, 
Paul  Meehl  (1954)  compiled  evidence  comparing  expert  clinical  prediction  with  actuarial 
methods  and  found  that  the  statistics  were  consistently  more  accurate  than  the  clinicians 
(as  cited  in  Gilovich  et  al,  2002,  p.  2).  His  research  also  uncovered  a  sharp  discrepancy 
between  the  clinicians’  assessments  of  their  performance  and  their  actual  record  of 
success.  This  juxtaposition  of  modest  perfonnance  and  robust  confidence  inspired  a  slew 

1  The  classic  theory  of  rationality  assumes  nothing  about  the  content  of  the  actor’s  objectives,  only  that 
whatever  those  objectives,  the  actor  has  reviewed  all  alternatives  and  accurately  assesses  all  consequences 
in  making  the  value-maximizing  choice  (Allison  &  Zelikow,  1999,  p.  18). 
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of  research  on  faulty  processes  of  reasoning  that  yield  compelling  yet  mistaken 
inferences.  In  what  became  a  key  methodological  contribution  by  introducing  Bayesian 
analysis  to  psychology,  Ward  Edwards  (1963)  provided  a  normative  standard  with  which 
everyday  judgments  could  be  compared.  From  his  research,  and  from  much  of  what 
followed,  it  became  clear  that  intuitive  judgments  of  likelihood  rarely  match  the 
nonnative  standard  (as  cited  in  Gilovich  et  al,  2002,  p.  2). 

2.  Bounded  Rationality 

Herbert  Simon  (1959)  became  the  first  to  directly  challenge  the  classical  model  of 
rational  choice,  noting  that  in  practice,  analysis  and  decision-making  is  less  than  fully 
rational;  it  is  constrained  by  inescapable  limitations  in  the  knowledge  and  computational 
ability  of  the  analyst  or  decision-maker.  He  proposed  a  more  limited  criterion  for  actual 
performance,  famously  dubbed  “bounded  rationality,”  that  acknowledged  the  inherent 
processing  limitations  of  the  human  mind.  Because  of  these  limits,  Simon  argued,  the 
mind  cannot  cope  directly  with  the  complexity  of  the  world.  Instead,  we  construct  a 
simplified  mental  model  of  reality  and  then  work  within  it.  And  although  we  behave 
rationally  within  the  confines  of  our  mental  model,  it  is  not  always  well  suited  to  the 
requirements  of  the  real  world  (as  cited  in  Heuer,  1999,  p.  3).2 

Like  its  classical  predecessor,  the  theory  of  bounded  rationality  has  important 
implications  for  analysis  and  decision-making.  If  we  accept  the  proposition  that  both  the 
knowledge  and  the  computational  power  of  the  analyst  or  decision-maker  are  severely 
limited,  we  must  distinguish  between  the  real  world  and  the  analyst’s  perception  of  and 
reasoning  about  it.  Rather  than  requiring  perfect  application  of  the  laws  of  logic  and 
probability,  this  more  limited  approach  suggests  that  analysts  or  decision-makers  reach 
conclusions  in  a  way  that  is  procedurally  reasonable  in  light  of  the  available  knowledge 
and  means  of  computation  (Simon,  1986,  p.  21 1). 3 

-  The  concept  of  bounded  rationality  has  come  to  be  recognized  widely,  though  not  universally,  both 
as  an  accurate  portrayal  of  human  judgment  and  choice  and  as  a  sensible  adjustment  to  the  limitations 
inherent  in  how  the  human  mind  functions  (Heuer,  1999,  p.  26). 

3  Simon  makes  a  further  distinction  between  classical  and  bounded  rationality.  To  deduce  the 
classically  rational  choice  in  a  given  situation,  we  need  only  know  the  choosing  organism’s  goals  and  the 
objective  characteristics  of  the  situation.  We  need  to  know  absolutely  nothing  else  about  the  organism.  In 
contrast,  to  deduce  the  boundedly  rational  choice  in  the  same  situation,  Simon  contends  we  must  know  the 
choosing  organism’s  goals,  the  information  and  the  conceptualization  it  has  of  the  situation,  and  its  abilities 
to  draw  inferences  from  the  information  it  possesses  (Simon,  1985,  p.  294). 
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B.  HEURISTICS  AND  BIASES 

Inspired  by  the  examples  of  Meehl,  Edwards,  Simon,  and  others,  Daniel 
Kahneman  and  Amos  Tversky  (1972)  developed  their  own  perspective  on  bounded 
rationality.  Although  they  fully  recognized  the  role  of  task  complexity  and  limited 
computational  capability  on  individual  perfonnance,  the  two  became  convinced  that  the 
processes  involved  in  judgment  and  choice  “were  not  merely  simpler  than  the  rational 
model  demanded,  they  were  categorically  different  in  kind”  (Gilovich  et  al,  2002,  p.  3). 

Acknowledging  a  “dual-processes”  approach  to  cognition,  Kahneman  and 
Tversky  (1972)  recognized  that  people  apprehend  reality  in  two  fundamentally  different 
ways.  The  first,  variously  labeled  intuitive,  automatic,  natural,  nonverbal,  narrative,  or 
experiential,  is  typically  thought  of  as  fast,  automatic,  effortless,  associative,  and  difficult 
to  control  or  modify.  The  second,  called  inferential,  analytical,  deliberate,  verbal,  or 
rational,  is  typically  thought  of  as  slower,  serial,  effortful,  deliberately  controlled,  and 
also  relatively  flexible  and  potentially  rule-governed  (Kahneman,  2002,  p.  450). 

Kahneman  and  Tversky  (1972)  also  suggested  that  people  rely  on  a  limited 
number  of  heuristic  principles  that  reduce  complex  inferential  tasks,  like  assessing 
probabilities  and  predicting  values,  into  simpler  judgmental  operations  (Kahneman,  2002, 
p.  465).  The  two  researchers  tied  these  heuristics  to  “natural  assessments,”  elicited  by  the 
task  at  hand,  that  influence  judgment  beneath  the  surface  of  consciousness.  They  noted, 
for  example,  that  when  deciding  whether  an  elegantly-dressed  lawyer  is  more  likely  to  be 
a  public  defender  or  a  member  of  a  large  corporate  firm,  we  cannot  help  computing  the 
similarity  between  the  individual  and  the  prototype  of  each  professional  niche. 
Kahneman  and  Tversky  (1972)  suggested  that  these  assessments  then  inform  such 
judgments  of  likelihood  in  the  absence  of  deliberate  intent  (Gilovich  et  al,  2002,  pp.  4-5). 
They  further  argued  that  while  heuristics  based  on  such  assessments  are  generally  quite 
useful,  they  often  lead  to  unconscious  biases  that  give  rise  to  severe  and  systematic 
errors. 

Kahneman  and  Tversky  (1972)  drew  three  overarching  conclusions  from  their 
work.  First,  the  two  claimed  that  although  these  heuristics  can  be  distinguished  from 
normative  reasoning  processes  by  patterns  of  biased  judgments,  the  heuristics  themselves 
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are  sensible  estimation  procedures  that  are  by  no  means  “irrational.”  Second,  they  argued 
that  although  these  heuristics  yield  “quick  and  dirty”  solutions,  they  draw  on  underlying 
processes  that  are  highly  sophisticated.  Finally,  the  two  argued  that  these  heuristics  are 
not  exceptional  responses  to  problems  of  excessive  complexity  or  an  overload  of 
information,  but  normal  responses  even  to  the  simplest  questions  about  likelihood, 
frequency,  and  prediction  (Gilovich  et  al,  2002,  p.  3). 

C.  SUMMARY 

In  the  thirty  years  since  Kahneman  and  Tversky  (1972)  introduced  the  concept, 
the  influence  of  heuristics  and  biases  has  gained  widespread  acceptance  both  within  and 
outside  the  psychological  community.  Yet  despite  the  admonitions  of  a  number  of 
visionaries,  this  appreciation  has  not  typically  extended  to  the  U.S.  intelligence 
community;  few  analysts  or  decision-makers  seem  to  fully  appreciate  the  extent  to  which 
heuristics  can  give  rise  to  systematic  error.  As  a  result,  many  of  their  judgments  have 
been  colored  by  unintentional  bias.4 

The  following  chapters  seek  to  illuminate  this  deficiency,  in  the  hope  that 
awareness  of  perceptual  and  cognitive  baggage  can  help  mitigate  the  severity  of  the 
errors  that  often  result  from  it.  Chapter  II  examines  the  perceptual  and  cognitive 
processes  on  which  judgment  is  based,  and  illustrates  how  while  necessary  and  often 
useful,  they  often  give  rise  to  error.  Chapter  III  builds  upon  this  foundation,  exploring 
the  heuristics  and  biases  first  introduced  by  Kahneman  and  Tversky  (1972)  more  than 
thirty  years  ago.  Chapter  IV  examines  the  concept  of  anchoring,  both  in  terms  of 
adjustment  and  activation,  and  suggests  how  it  often  compounds  the  errors  caused  by 
bias.  Chapter  V  looks  at  a  specific  case,  the  Intelligence  Community’s  assessment 
concerning  Iraqi  Weapons  of  Mass  Destruction,  in  order  to  demonstrate  that  an 
appreciation  of  heuristics  and  biases  is  more  than  academic,  and  that  shortcomings  in  the 
perceptual  and  cognitive  processes  that  underlie  judgment  can  have  the  most  serious  of 
consequences. 


4  Richard  Heuer  (1999),  a  career  analyst  for  the  Central  Intelligence  Agency,  notes  that  the  impact  of 
such  bias  is  most  pronounced  in  the  very  conditions  “under  which  intelligence  analysis  is  typically 
conducted  -  dealing  with  highly  ambiguous  situations  on  the  basis  of  information  that  is  processed 
incrementally  under  pressure  for  early  judgment”  (p.  14). 
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II.  FUNDAMENTALS  OF  PERCEPTION  AND  COGNITION 


Human  beings  construct  our  own  version  of  reality  using  information  provided  by 
our  senses.  Yet  sensory  input  is  mediated  by  complex  psychological  processes  that 
determine  the  manner  in  which  we  select,  organize,  and  attribute  meaning  and  value  to 
information.  What  people  perceive,  how  readily  we  perceive  it,  and  how  we  use  this 
information  to  make  judgments  and  decisions  are  all  clearly  influenced  by  past 
experience,  education,  cultural  values,  role  requirements,  and  organizational  norms,  as 
well  as  by  the  specifics  of  the  infonnation  and  the  environment  in  which  it  is  received 
(Heuer,  1999,  p.  4). 

Yet  our  decisions  are  also  shaped  by  heuristics  and  biases  that  operate  apart  from 
any  experiential  or  environmental  dynamics,  beneath  the  surface  of  our  consciousness. 
As  the  following  sections  describe,  a  number  of  processes  essential  to  perception  and 
cognition  can  also  often  give  rise  to  systematic  errors  in  judgment. 

A.  PERCEPTION 

The  process  of  perception  links  people  to  our  environment  and  is  critical  to 
understanding  the  world  around  us.  Cogent  analysis  and  effective  decision-making 
clearly  begin  with  and  depend  on  accurate  perception.  Yet  research  into  human 
perception  demonstrates  that  the  process  is  beset  with  pitfalls  (Heuer,  1999,  p.  7). 

1.  Perception  is  Unavoidably  Selective 

People  often  think  of  perception  as  an  objective  process.  “We  see,  hear,  smell, 
taste  or  feel  stimuli  that  impinge  upon  our  senses”  (Heuer,  1999,  p.  7).  We  think  that  if 
we  are  careful  and  attentive,  we  are  able  to  record  what  actually  exists.  Yet  a  growing 
body  of  evidence  suggests  that  perception  is  an  active  passive  process  that  constructs 
rather  than  records  reality. 

Bottlenecks  in  our  mental  machinery  admit  only  a  small  fraction  of  what  actually 
exists;  we  are  simply  unable  to  process  all  that  there  is  to  see.  As  a  result,  some  view 
perception  as  an  approximation  of  reality,  in  which  the  subjective  world  of  the  decision¬ 
maker  resembles  the  external  environment  closely,  but  lacks,  perhaps,  some  fineness  of 
detail  (Simon,  1959,  p.  272).  Others  view  perception  as  a  filtered  version  of  reality,  in 
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which  only  a  small  subset  of  what  actually  exists  enters  the  central  nervous  system.  Y et 
studies  have  shown  that  the  perceived  world  is  markedly  different  from  the  real  world. 
Perception  is  neither  a  crude  approximation  nor  some  passive  selection  of  part  of  a 
presented  whole;  it  is  “a  deliberate  process  involving  attention  to  a  very  small  part  of  the 
whole  and  exclusion  of  almost  all  that  is  not  within  the  scope  of  attention”  (p.  273). 

Consider  the  following  description  of  a  common  perceptual  field: 

I  am  standing  at  the  window  and  see  a  house,  trees,  sky.  And  now,  for 
theoretical  purposes,  I  could  try  to  count  and  say:  there  are  .  .  .  327 
nuances  of  brightness  and  hue.  Do  I  see  327?  No;  I  see  sky,  house,  [and] 
trees  (Wertheimer  as  cited  in  Woods  et  al,  2002,  p.  28). 

Wertheimer’s  realization  reflects  a  fundamental  aspect  of  human  perception: 
Perception  involves  selection,  simplification,  reduction,  and  omission.  If  we  count  the 
basic  elements  in  the  perceptual  field  described  above,  for  example,  we  find  an 
overwhelming  number  varying  in  hue,  saturation  and  brightness.  Yet  this  avalanche  of 
data  does  not  incapacitate  us.  Perceptual  organization  reduces  the  stimulus  data, 
“grouping  a  large  number  of  picture  elements  into  a  small  number  of  seen  objects  and 
their  parts”  (Goldmeier  as  cited  in  Woods  et  al,  2002,  p.  28).  It  reduces  a  complex  field 
into  a  few  objects  or  events  and  the  relationships  between  them. 

2.  Perception  is  Influenced  by  Expectations 

People  develop  expectations  from  a  myriad  of  diverse  sources,  including  past 
experience,  professional  training,  and  cultural  and  organizational  norms.  All  these 
influences  predispose  us  to  pay  particular  attention  to  certain  kinds  of  infonnation  and  to 
organize  and  interpret  it  in  certain  ways  (Heuer,  1999,  p.  34). 

A  number  of  experiments  have  been  conducted  to  show  the  extraordinary  extent 
to  which  what  we  perceive  depends  upon  our  own  unconscious  assumptions  and 
preconceptions.  “What  people  perceive,  and  how  readily  they  perceive  it,  are  strongly 
influenced  by  their  past  experience,  education,  cultural  values,  and  role  requirements,  as 
well  as  by  the  stimuli  recorded  by  their  receptor  organs”  (Heuer,  1999,  p.  7).  Consider 
the  following  figure: 
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Figure  1.  Perception  and  Expectations  (Heuer,  1999,  p.  8) 

Most  people  overlook  that  in  each  of  the  three  phrases,  the  definite  (or  indefinite) 
article  is  repeated.  This  simple  exercise  demonstrates  a  second  fundamental 
characteristic  of  human  perception:  We  tend  to  perceive  what  we  expect  to  perceive 
(Heuer,  1999,  p.  8). 

In  an  experiment  investigating  this  phenomenon,  Jerome  Bruner  and  Leo  Postman 
(1949)  showed  subjects  a  series  of  playing  cards,  some  of  which  were  altered  so  that  the 
spades  were  red  and  the  hearts  black.  Each  card  was  presented  successively,  for  varying 
periods  of  time,  until  correct  recognition  occurred.  The  two  researchers  made  a  number 
of  key  observations.  First,  the  subjects  identified  the  nonnal  cards  more  quickly  and 
accurately  than  the  anomalous  ones.  Second,  after  realizing  that  some  of  the  cards  had 
been  altered,  the  subjects  improved  their  overall  speed  and  accuracy,  but  only 
moderately.  Bruner  and  Postman  concluded  that  perceptual  organization  is  powerfully 
determined  by  expectations  built  upon  past  interaction  with  the  environment.  When  such 
expectations  are  violated,  the  perceiver  resists  recognition  of  the  unexpected  (as  cited  in 
Chandler,  2004). 

3.  Perception  is  Shaped  by  Situational  Context 

Different  circumstances  evoke  different  sets  of  expectations.  We  are  more 
attuned  to  hearing  footsteps  behind  us  when  walking  in  an  alley  at  night,  for  example, 
than  along  a  city  street  in  daytime.  Moreover,  the  meaning  we  attribute  to  the  sound  of 
footsteps  will  vary  under  these  differing  circumstances  (Heuer,  1999,  p.  9).  This  simple 
analogy  reflects  a  third  characteristic  of  human  perception:  We  tend  to  perceive  objects 
or  events  differently  based  on  the  situation.  In  other  words,  perception  is  influenced  by 
the  context  in  which  it  occurs. 

In  an  experiment  investigating  this  phenomenon,  B.R.  Bugelski  and  Delia 
Alampay  (1961)  presented  subjects  with  a  drawing  that  could  be  interpreted  either  as  rat 
or  an  old  man  wearing  spectacles: 
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Figure  2.  Old  man  or  rat?  (Chandler,  2004) 


Prior  to  seeing  this  image,  half  of  the  group  was  shown  drawings  of  various  animals: 
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Figure  3.  Animal  series  (Chandler,  2004) 


The  other  half  of  the  group  was  shown  drawings  of  human  faces: 


Figure  4.  Faces  series  (Chandler,  2004) 


A  third  (control)  group  was  shown  no  pictures  beforehand.  The  two  researchers  made  a 
number  of  key  observations.  More  than  80%  of  the  control  group  characterized  the 
ambiguous  image  as  a  man  rather  than  a  rat.  Yet  the  group  that  was  first  shown  the 
drawings  of  animals  consistently  characterized  the  ambiguous  drawing  as  a  rat  rather  than 
a  man.  In  fact,  the  more  animals  that  they  were  shown  before  seeing  the  ambiguous 
image,  the  more  likely  they  were  to  see  a  rat.  Bugelski  and  Alampay  (1961)  concluded 
that  perception  depends  not  only  on  the  objective  qualities  of  an  object  or  event,  but  also 
on  the  condition  or  situation  in  which  the  object  is  found  (as  cited  in  Chandler,  2004). 

4.  Perception  is  Influenced  by  Blurred  or  Ambiguous  Stimuli 

Under  ordinary  circumstances,  perception  operates  relatively  effortlessly.  If  the 
clarity  of  the  information  presented  is  diminished  in  some  way,  however,  recognition 
takes  longer  (Bruner  &  Potter,  1964,  p.  424).  A  number  of  studies  suggest  that  if  a 
subject  is  initially  exposed  to  a  blurred  or  otherwise  diminished  image  that  he  cannot 
recognize,  subsequent  recognition  of  the  image  in  clearer  form  is  substantially  delayed. 
These  studies  reflect  a  fourth  fundamental  characteristic  of  human  perception:  initial 
exposure  to  blurred  or  ambiguous  stimuli  interferes  with  accurate  perception  even  after 
more  and  better  infonnation  becomes  available  (Heuer,  1999,  p.  37). 
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In  one  such  study,  Jerome  Bruner  and  Mary  Potter  (1964)  showed  subjects  eight 
ordinary  color  photographs  of  common  everyday  objects,  projected  one  at  a  time.  Each 
photo  was  initially  exposed  in  a  state  of  blur  and  brought  continuously  into  better  focus  in 
order  to  detennine  at  what  point  the  subjects  could  identify  them  correctly  (Bruner  & 
Potter,  1964,  p.  424).  The  researchers  made  two  key  observations.  First,  they  found  that 
those  who  began  viewing  when  the  photos  were  most  out  of  focus  had  substantially  more 
difficulty  identifying  them  as  they  became  clearer  than  those  who  started  viewing  when 
the  photos  were  less  blurred.  In  other  words,  the  greater  the  initial  blur,  the  clearer  the 
picture  had  to  be  before  people  could  recognize  it.  Second,  the  longer  people  were 
exposed  to  a  blurred  picture,  the  clearer  the  picture  had  to  be  before  they  could  recognize 
it  (Heuer,  1999,  p.  14). 

Bruner  and  Potter  (1964)  concluded  that  what  happened  in  this  experiment  is  what 
presumably  happens  in  real  life:  despite  ambiguous  stimuli,  people  form  tentative 
hypotheses  about  what  we  see.  The  longer  we  are  exposed  to  an  ambiguous  image,  the 
greater  confidence  we  develop  in  our  initial  impression  of  it.  Even  if  this  impression  is 
later  proven  to  be  wrong,  it  still  has  a  significant  impact  on  subsequent  perceptions.  “For 
a  time,  as  the  picture  becomes  clearer,  there  is  no  obvious  contradiction;  the  new  data  are 
assimilated  into  the  previous  image,  and  the  initial  interpretation  is  maintained  until  the 
contradiction  becomes  so  obvious  that  it  forces  itself  upon  our  consciousness”  (Heuer, 
1999,  p.  14). 

B.  COGNITION 

Because  of  limits  in  our  mental  capacity,  the  human  mind  cannot  cope  directly 
with  the  complexity  of  the  world.  Rather,  we  construct  simplified  mental  models  of 
reality  and  then  work  with  them  (Heuer,  1999,  p.  3).  Yet  people  rarefy  enter  into  the 
sense-making  endeavor  with  a  clean  slate.  We  bring  with  us  a  set  of  theoretical 
predispositions  that  enter  the  process  as  mediating  variables.  These  predispositions 
greatly  aid  our  understanding  of  events  and  the  construction  of  cause  and  effect 
attributions,  “keeping  us  from  having  to  reinvent  the  wheel  at  every  decision-making 
juncture”  (Bukszar,  1999,  p.  108). 

Like  the  processes  involved  in  perception,  our  cognitive  models  influence  our 
judgment.  They  play  a  central  and  unifying  role  in  representing  objects,  states  of  affairs, 
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sequences  of  events,  the  way  the  world  is,  and  the  social  and  psychological  actions  of 
daily  life”  (Johnson-Laird,  1993,  p.  397).  They  allow  us  to  make  inferences  and 
predictions,  to  understand  phenomena,  to  experience  events  by  proxy,  and  to  decide  what 
action  to  take  and  to  control  its  execution  (p.  397).  Yet  the  processes  involved  in  the 
construction  and  employment  of  cognitive  models,  like  those  fundamental  to  perception, 
are  also  often  beset  with  pitfalls. 

1.  Mental  Models  are  Unavoidable 

Some  people  view  mental  models  as  something  bad,  to  be  avoided.  They  argue 
that  people  should  have  an  open  mind  and  be  influenced  only  by  the  facts  rather  than  by 
preconceived  notions.  Yet  this  ideal  is  unreachable.  There  is  no  such  thing  as  “just  the 
facts”  of  the  case.  “There  is  only  a  very  selective  subset  of  the  overall  mass  of  data  to 
which  one  has  been  subjected  that  one  takes  as  facts  and  judges  to  be  relevant  to  the 
question  at  hand”  (Heuer,  1999,  p.  10). 

In  fact,  mental  models  are  unavoidable.  The  world,  from  the  perspective  of  the 
human  mind,  is  fundamentally  uncertain.  People  have  no  conceivable  way  of  coping 
with  the  volume  of  stimuli  that  impinge  upon  our  senses,  or  with  the  volume  and 
complexity  of  the  data  we  have  to  analyze,  without  some  kind  of  simplifying 
preconceptions  about  what  to  expect,  what  is  important,  and  what  is  related  to  what 
(Heuer,  1999,  p.  10).  Although  our  brain  does  not  often  have  sufficient  information  to 
know  for  certain  what  is  out  there,  it  is  not  paralyzed  by  uncertainty.  Instead,  it  evaluates 
its  environment  and  uses  models  to  make  a  “best  guess”  (Gigerenzer,  2005,  p.  197). 

Consider  the  following  illustration: 


Figure  5.  Convex  or  Concave  (Gigerenzer,  2005) 

The  dots  on  the  left  side  appear  concave,  receding  into  the  surface  away  from  the 
observer.  The  dots  on  the  right  appear  convex,  rising  up  from  the  surface  toward  the 
observer.  When  the  page  is  turned  upside  down,  however,  the  concave  dots  turn  into 
convex  dots,  and  vice  versa  (Gigerenzer,  2005,  p.  197). 
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There  is,  of  course,  no  third  dimension,  and  there  are  no  convex  or  concave  dots. 
Our  recognition  of  concavity  or  convexity  is  the  result  of  a  basic  mental  model.  D.A. 
Kleffner  and  Vilayanur  Ramachandran  (1992)  found  that  our  brain  assumes  a  three- 
dimensional  world  and  uses  the  shaded  parts  of  the  dots  to  guess  in  what  direction  of  the 
third  dimension  they  extend.  By  experimentally  varying  factors  such  as  the  location  of 
the  light  source  and  the  shading,  and  documenting  their  effect  on  the  illusion,  they 
concluded  that  humans  assume  a  single  source  of  light  from  above.  The  two  concluded 
that  these  ecological  structures  arise  from  our  collective  evolution,  where  the  sun  and 
moon  were  the  only  sources  of  light,  and  only  one  operated  at  a  time.  “The  brain  exploits 
these  assumed  structures  by  using  a  fast  and  frugal  heuristic:  If  the  shade  is  in  the  upper 
part,  then  the  dots  are  concave;  if  the  shade  is  in  the  lower  part,  then  the  dots  are  convex” 
(as  cited  in  Gigerenzer,  2005,  p.  197). 

2.  Mental  Models  are  Quick  to  Form  but  Resistant  to  Change 
The  cognitive  models  that  underlie  judgment  and  decision  are  often  quick  to  fonn, 
yet  highly  resistant  to  change.  Consider  the  following  image:5 


Old  or  young  woman?  (Chandler,  2004) 

This  now-famous  drawing,  designed  to  be  interpreted  as  either  a  young  or  an  old  woman, 
was  first  introduced  into  the  psychological  literature  by  Edwin  Boring  (1930). 6  In  an 
experiment  on  sensory  organization  and  learning,  Robert  Leeper  (1935)  had  the  image 
redrawn  in  two  less  ambiguous  fonns:  one  which  emphasized  the  old  woman  and  the 
other  which  emphasized  the  young  woman: 


5  The  old  woman’s  nose,  mouth,  and  eye  are,  respectively,  the  young  woman’s  chin,  necklace,  and 
ear.  Both  are  seen  in  profile  looking  left;  we  see  the  young  woman  mainly  from  behind  so  most  facial 
features  are  not  visible.  Her  eyelash,  nose,  and  the  curve  of  her  cheek  may  be  seen  just  above  the  old 
woman’s  nose  (Heuer,  1999,  p.  13). 

6  The  image  was  published  by  the  British  cartoonist  W.  E.  Hill  in  1915,  and  is  thought  to  be  based  on 
a  French  version  of  15  years  earlier.  It  is  sometimes  referred  to  as  “The  Wife  and  the  Mother-in-Law” 
(http://www.aber.ac.uk/media/Modules/MC10220/visper05.html,  n.d.). 
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Figure  6.  Old  or  young  woman:  less  ambiguous  forms  (Chandler,  2004) 


Leeper  enlisted  a  number  of  subjects  and  divided  them  into  five  groups.  A  control  group 
was  shown  only  the  ambiguous  drawing;  65  percent  described  the  image  as  that  of  a 
young  woman.  The  second  and  third  groups  were  given  a  verbal  description  of  the  old 
woman  and  young  woman  respectively,  before  being  shown  any  image.  The  fourth  and 
fifth  groups  were  shown  the  “old”  version  and  the  “young”  version  respectively.  All  of 
the  groups  (other  than  the  control  group)  were  then  shown  the  original  ambiguous  image. 
An  overwhelming  majority  of  the  subjects  in  the  groups  that  saw  or  were  told  about  the 
old  woman  described  the  ambiguous  version  as  “old.”  All  of  the  subjects  in  the  groups 
that  saw  or  were  told  about  the  young  woman  described  the  ambiguous  version  as 
“young.”  Leeper  concluded  that  once  the  subjects  had  formed  a  mental  model,  they 
found  changing  it  very  difficult  (as  cited  in  Chandler,  2004). 

3.  New  Information  is  Assimilated  to  Existing  Content 

Human  beings  constantly  create  or  construct  new  mental  models;  the  content  of 
our  minds  is  inherently  open  and  difficult  to  define.  Yet  at  the  same  time,  we  itemize  and 
categorize  this  content  in  order  to  systemize  the  vast  amount  of  data.  “A  tension 
therefore  exists  between  our  desire  to  update  our  mental  models  constantly  versus  our 
need  to  categorize  and  store  (or  delete)  our  knowledge”  (Karp,  2005,  p.  7). 

Consider  the  following  illustration: 


Figure  7.  Man  or  woman?  (Heuer,  1999) 


The  figure,  developed  by  Gerald  Fisher  in  1967  as  part  of  an  experiment  on  ambiguity,  is 
part  of  a  series  of  progressively  modified  drawings  that  change  almost  imperceptibly 
from  a  man  into  a  woman:  7 


Figure  8.  Man  transfonning  into  woman  (Heuer,  1999) 

When  viewed  alone,  half  of  Fisher’s  subjects  characterized  the  initial  image  as  a  man;  the 
other  half  as  a  woman.  When  subjects  were  shown  the  entire  series  of  drawings  one  by 
one,  their  perception  of  this  drawing  was  biased  according  to  which  end  of  the  series  they 
started  from.  Subjects  who  started  by  viewing  a  picture  that  was  clearly  a  man  were 
biased  in  favor  of  continuing  to  see  a  man  long  after  an  objective  observer  recognized 
that  the  man  had  “become”  a  woman.  Similarly,  test  subjects  who  started  at  opposite  end 
of  the  series  were  biased  in  favor  of  continuing  to  see  a  woman  (Heuer,  1999,  p.  10). 

The  experiment  demonstrates  that  once  an  observer  forms  a  mental  model 
concerning  the  phenomenon  being  observed,  it  conditions  future  characterizations  of  that 
phenomenon.  This  also  explains  why  gradual,  evolutionary  change  often  goes  unnoticed 
(Heuer,  1999,  p.  11).  Moreover,  the  tendency  to  assimilate  new  data  into  pre-existing 
images  is  greater  “the  more  ambiguous  the  infonnation,  the  more  confident  the  actor  is  of 
the  validity  of  his  image,  and  the  greater  his  commitment  to  the  established  view”  (p.  1 1). 


7  In  the  series,  the  initial  image  occupies  the  right-most  position  in  the  top  row. 
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III.  FUNDAMENTALS  OF  PERCEPTION  AND  COGNITION 


Uncertainty  is  an  unavoidable  aspect  of  the  human  condition  (Gilovich  et  al, 
2002,  p.  19).  Many  of  the  choices  we  make  are  necessarily  based  on  beliefs  or  best 
guesses  about  the  likelihood  of  uncertain  events  such  as  the  guilt  of  a  defendant,  the 
result  of  an  election,  the  future  value  of  the  dollar,  the  outcome  of  a  medical  operation,  or 
the  response  of  a  friend.  “Because  we  normally  do  not  have  adequate  formal  models  for 
computing  the  actual  probabilities  of  such  events,  intuitive  judgment  is  often  the  only 
practical  method  for  assessing  uncertainty”  (Kahneman  &  Tversky,  1984,  p.  293). 

Yet  human  intuition  is  far  from  perfect.  In  contrast  to  fonnal  theories  or  beliefs, 
intuitive  judgments  of  probability  come  neither  from  the  rules  of  logic  nor  the 
mathematical  laws  governing  probability.  People  do  not  normally  analyze  daily  events 
into  exhaustive  lists  of  possibilities  or  evaluate  compound  probabilities  by  aggregating 
elementary  ones.  Instead,  we  employ  a  number  of  heuristics,  “based  on  natural 
assessments  that  are  routinely  carried  out  as  part  of  the  perception  of  events  and  the 
comprehension  of  messages”  (Gilovich  et  al,  2002,  p.  20).  For  example: 

The  mere  mention  of  “horror  movies”  activates  instances  of  horror  movies 
and  evokes  an  assessment  of  their  availability.  Similarly,  the  statement 
that  Woody  Allen’s  aunt  had  hoped  that  he  would  be  a  dentist  elicits  a 
comparison  of  the  character  to  the  stereotype  and  an  assessment  of 
representativeness.  It  is  presumably  the  mismatch  between  Woody 
Allen’s  personality  and  our  stereotype  of  a  dentist  that  makes  the  thought 
mildly  amusing  (Gilovich  et  al,  2002,  p.  20). 

Although  such  assessments  are  based  neither  on  frequency  nor  probability,  they  often 
play  a  dominant  role  when  intuitive  judgments  are  required.  “The  availability  of  horror 
movies  may  be  used  to  answer  the  question,  ‘What  proportion  of  the  movies  produced 
last  year  were  horror  movies?’,  and  representativeness  may  control  the  judgment  that  a 
particular  boy  is  more  likely  to  be  an  actor  than  a  dentist”  (Gilovich  et  al,  2002,  p.  20). 
The  following  sections  examine  these  two  powerful  heuristics  and  the  biases  which  they 
give  rise  to  in  greater  detail. 
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A.  REPRESENTATIVENESS 

One  of  the  most  basic  and  most  often  employed  natural  assessment  is  the 
representativeness  heuristic.  It  evaluates  the  degree  of  correspondence  between  a  sample 
and  a  population,  and  instance  and  a  category,  an  act  and  an  actor,  or  an  outcome  and  a 
model  (Gilovich  et  al,  2002,  p.  22).  It  allows  people  to  reduce  otherwise  taxing 
inferential  tasks  into  simple  similarity  judgments,  in  which  objects,  events,  or  processes 
are  assigned  to  one  conceptual  category  rather  than  to  another  based  on  how  well  they 
represent  or  resemble  one  category  more  than  another  (Nisbett  &  Ross,  1980,  p.  7).  We 
use  the  representativeness  heuristic  everyday  when  we  shop  for  groceries,  empty  the 
dishwasher,  or  find  a  place  to  park  our  car.  “Countless  tasks,  especially  those  requiring 
induction  or  generalization,  depend  on  deciding  what  class  or  category  of  event  one  is 
observing.  Such  judgments  invariably  hinge  on  assessments  of  resemblance  or 
representation”  (Nisbett  &  Ross,  1980,  p.  27). 

Problems  with  representativeness  arise  when  we  use  it  as  our  primary  (or  our 
only)  judgmental  strategy.  “This  useful  heuristic  becomes  an  undesirable  bias  when  we 
overestimate  the  extent  to  which  a  situation  or  sample  is  representative  of  the  situation  or 
population  to  which  we  wish  to  generalize”  (Schwenk,  1985,  p.  121).  For  example, 
“people  express  great  confidence  in  the  prediction  that  a  person  is  a  librarian  when  a 
given  description  of  his  personality  which  matches  the  stereotype  of  librarians,  even  if  the 
description  is  scanty,  unreliable,  or  outdated”  (Kahneman  et  al,  1982,  p.  9).  When,  for 
whatever  reason,  categorization  based  on  the  known  features  of  an  object  or  event  is  not 
possible,  statistical  considerations  should  take  over.  “In  particular,  the  relative  frequency 
of  the  categories  in  the  population  under  consideration  [should]  become  the  normatively 
appropriate  guide  to  categorization  to  the  extent  that  the  known  features  of  the  object  are 
ambiguous  guides  to  categorization”  (Nisbett  &  Ross,  1980,  p.  7).  Yet  few  people 
understand  these  statistical  considerations,  or  how  to  combine  them  with 
representativeness  considerations . 

Thomas  Gilovich  (1981)  conducted  a  number  of  studies  examining 
representativeness  bias.  In  one,  students  enrolled  in  a  political  science  class  were  asked 
to  formulate  U.S.  foreign  policy  for  a  hypothetical  international  crisis.  The  students  were 
divided  into  three  groups;  each  group  was  given  scenarios  containing  information 
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irrelevant  to  the  outcome  of  the  crisis.  One  scenario  was  reminiscent  of  World  War  II; 
the  other  of  Vietnam.  The  results  were  dramatic.  The  World  War  II  group 
overwhelming  favored  an  interventionist  strategy,  while  the  Vietnam  group 
overwhelming  favored  an  isolationist  approach.  Gilovich  concluded  that  his  subjects 
chose  a  course  of  action  based  solely  on  its  representativeness  to  the  scenario  they  were 
given,  despite  the  fact  that  they  had  better  infonnation  with  which  to  make  a  decision  (as 
cited  in  Stormer,  1991,  p.  8). 

1.  Representativeness  and  the  Misconception  of  Chance 

People  often  expect  that  a  sequence  of  events  generated  by  a  random  process  will 
represent  the  essential  characteristics  of  that  process  even  when  the  sequence  is  short 
(Kahneman  &  Tversky,  1982,  p.  7).  In  considering  a  series  of  coin  tosses,  for  example, 
people  regard  the  sequence  H-T-H-T-T-H  to  be  more  likely  than  the  sequence  H-H-H-T- 
T-T,  which  does  not  appear  as  random;  and  also  more  likely  than  the  sequence  H-H-H-H- 
T-H,  which  does  not  represent  the  fairness  of  the  coin.  “People  expect  that  the  essential 
characteristics  of  the  process  will  be  represented,  not  only  globally  in  the  entire  sequence, 
but  also  locally  in  each  of  its  parts”  (Kahneman  et  al,  1982,  p.  7). 8 

The  “gambler’s”  fallacy,  the  mistaken  belief  that  a  successful  outcome  is  due  after 
a  run  of  bad  luck,  is  another  example  of  the  misconception  of  chance.  After  observing  a 
long  run  of  “red”  on  a  roulette  wheel,  for  example,  people  believe  that  “black”  is  due, 
because  the  occurrence  of  black  would  make  the  overall  sequence  of  events  more 
representative  of  the  generating  process  than  would  the  occurrence  of  another  red  (Pious, 
1993,  p.  1 13).  The  same  dynamic  may  be  seen  in  the  following  scenario: 

Suppose  that  an  unbiased  coin  is  flipped  three  times,  and  each  time  the 
coin  lands  on  Heads.  If  you  had  to  bet  $100  on  the  next  toss,  would  you 
choose  Heads  or  Tails?  (Pious,  1993,  p.  1 13). 

Because  the  coin  is  unbiased,  the  normatively  correct  answer  is  that  there  should  be  no 
preference  between  Heads  and  Tails.  Scott  Pious  (1993)  suggests,  however,  that  most 
people  believe  Tails  is  more  probable  after  a  run  of  three  Heads  (p.  1 13). 


8  Of  course,  a  locally  represented  sequence  like  the  coin  toss  described  above  deviates  systematically 
from  chance  expectation:  it  contains  too  many  alterations  and  too  few  runs  (Kahneman  et  al,  1982,  p.  7). 
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Belief  in  the  “law  of  small  numbers”  is  another  fonn  of  the  misconception  of 
chance  (Kahneman  et  al,  1982,  p.  9).  A  tongue-in-cheek  reference  to  the  law  of  large 
numbers,  the  basic  statistical  principle  stating  very  large  samples  will  be  highly 
representative  of  the  population  from  which  they  are  drawn;  the  law  of  small  numbers 
describes  our  tendency  to  place  too  much  reliance  on  small  samples.  Even  among 
professional  psychologists,  this  bias  has  shown  to  result  in  “seriously  incorrect  notions 
about  the  amount  of  error  and  unreliability  inherent  in  small  samples  of  data,  unwarranted 
confidence  in  the  early  trends  from  the  first  few  data  points,  and  unreasonably  high 
expectations  of  being  able  to  repeat  the  same  experiment  and  get  the  same  results  with  a 
different  set  of  test  subjects”  (Heuer,  1999,  p.  121). 

2.  Representativeness  and  the  Illusion  of  Validity  and  Order 
People  often  predict  by  selecting  the  outcome  that  is  most  representative  of  the 
input.  The  confidence  they  have  in  their  prediction  depends  primarily  upon  the  degree  of 
representativeness  (that  is,  the  quality  of  the  match  between  the  selected  outcome  and  the 
input),  with  little  or  no  regard  for  the  factors  that  limit  predictive  accuracy  (Kahneman  et 
al,  1982,  p.  9).  Consider  the  following  example: 

Tom  is  of  high  intelligence,  although  lacking  in  true  creativity.  He  has  a 
need  for  order  and  clarity,  and  for  neat  and  tidy  systems  in  which  every 
detail  finds  its  appropriate  place.  His  writing  is  rather  dull  and 
mechanical,  occasionally  enlivened  by  somewhat  corny  puns  and  by 
flashes  of  imagination  of  the  science  fiction  type  (Kahneman  et  al,  1982, 
p.  49). 

Daniel  Kahneman  and  Amos  Tversky  (1973)  asked  a  group  of  graduate  students 
to  read  this  description  of  Tom  and  then  predict  in  which  of  nine  subject  areas,  ranging 
from  computer  science  to  humanities  or  education,  Tom  would  most  likely  specialize. 
The  two  researchers  found  that  the  judgments  of  the  graduate  students  drastically  violated 
the  normative  rules  of  prediction.  More  than  95  percent  predicted  that  Tom  was  more 
likely  to  study  computer  science  than  humanities  or  education,  even  though  they  knew 
there  were  significantly  more  students  in  the  latter  field.  This  unwarranted  confidence, 
spawned  by  an  apparent  fit  between  the  input  information  and  the  predicted  outcome, 
produced  an  “illusion”  of  validity  (as  cited  in  Kahneman  et  al,  1982,  p.  9). 
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Paul  Slovic  (1966)  found  that  the  perceived  consistency  or  coherence  of  input  is  a 
related  determinant  of  representativeness.  “The  more  consistent  the  input,  the  more 
representative  the  prediction  will  appear  and  the  greater  the  confidence  [people  have]  in 
that  prediction”  (as  cited  in  Kahneman  et  al,  1982,  p.  65).  In  a  series  of  experiments 
examining  this  phenomenon,  Slovic  found  that  when  subjects  were  asked  to  predict  the 
final  grade-point  average  of  a  student,  they  were  much  more  confident  when  the  student’s 
previous  record  consisted  entirely  of  “B’s”  than  when  it  included  a  mix  of  “A’s”  and 
“C’s.”  This  intuition  that  orderly  or  consistent  profiles  allow  greater  predictability  than 
inconsistent  ones  is  compelling.  Yet  it  also  happens  to  be  wholly  incompatible  with  the 
laws  of  statistics.9  “Thus,  redundancy  among  inputs  decreases  accuracy  even  when  as  it 
increases  confidence,  and  people  are  often  confident  in  their  predictions  that  are  quite 
likely  to  be  off  the  mark”  (Kahneman  et  al,  1982,  p.  9). 

3.  Representativeness  and  the  Perception  of  Cause  and  Effect 

Representativeness  can  also  manifest  itself  in  our  perception  of  cause  and  effect. 
We  cannot  see,  of  course,  cause  and  effect  in  the  same  sense  that  we  see  a  desk  or  a  tree. 
“Even  when  we  observe  one  billiard  ball  striking  another  and  then  watch  the  previously 
stationary  ball  begin  to  move,  we  are  not  perceiving  cause  and  effect”  (Heuer,  1999,  p. 
127).  The  conclusion  that  one  ball  caused  the  other  to  move  results  not  from  direct 
sensory  perception,  but  rather  from  a  complex  process  of  inference  that  involves 
attributing  an  effect  to  the  object  or  event  that  caused  it. 

Yet  because  we  do  not  have  an  intuitive  understanding  of  the  kinds  and  amount  of 
information  needed  to  prove  a  relationship,  people  often  perceive  relationships  that  do 
not  in  fact  exist  (Heuer,  1999,  p.  127).  As  early  as  the  mid-nineteenth  century,  British 
philosopher  John  Stuart  Mill  proposed  that  humans  often  demonstrate  a  deep  seated 
fallacy  in  casual  reasoning:  “the  prejudice  that  the  conditions  of  a  phenomenon  must 
resemble  that  phenomenon”  (Mill,  1974,  p.  765).  More  than  150  years  later,  Richard 
Nisbett  and  Thomas  Wilson  (1977),  restating  Mill’s  proposal,  suggested  that  causal 
explanations  are  often  strongly  influenced  by  a  primitive  version  of  the 
representativeness  heuristic.  “People  have  strong  a  priori  notions  of  the  types  and  causes 

9  Specifically,  this  belief  is  incompatible  with  the  commonly  applied  multivariate  model  of  prediction 
in  which  expected  predictive  accuracy  is  independent  of  within-profile  variability  (Kahneman  et  al,  1982, 
p.  65). 
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that  ought  to  be  linked  to  particular  types  of  effects,  and  a  simple  ‘resemblance  criterion’ 
often  figures  heavily  in  such  notions”  (as  cited  in  Nisbett  &  Ross,  1982,  p.  115). 

In  a  classic  experiment  examining  this  phenomenon,  Lauren  and  Jean  Chapman 
(1967)  surveyed  clinicians  who  were  active  in  diagnostic  testing.  The  two  researchers 
wrote  brief  descriptions  of  six  types  of  patients  and  asked  each  clinician  which 
characteristics  the  patient  might  display  in  a  common  diagnostic  tool  known  as  the  Draw- 
A-Person  Test.  The  Chapmans  found  surprising  agreement  among  the  clinicians.  For 
example,  91  percent  said  that  a  patient  who  was  generally  suspicious  of  others  would 
draw  a  person  with  large  or  atypical  eyes,  and  82  percent  said  that  a  patient  concerned 
with  his  own  intelligence  would  draw  a  person  with  a  large  or  emphasized  head.  Even  at 
the  time  of  the  experiment,  however,  the  Draw-A-Person  test  had  been  proven  to  have  no 
validity.  The  correlations  that  the  clinicians  made  were  purely  illusory:  the  result  of 
shared  clinical  stereotypes  (as  cited  in  Pious,  1993,  p.  165). 

B.  AVAILABILITY 

When  people  are  required  to  judge  the  frequency  of  an  object  or  the  likelihood  of 
an  event,  we  are  often  influenced  by  its  availability  in  the  processes  of  perception, 
memory,  or  construction  from  imagination  (Nisbett  &  Ross,  1980,  p.  18).  In  particular, 
psychologists  have  shown  that  cues  such  as  the  ease  with  which  a  scenario  or  event 
comes  to  mind,  and  the  frequency  of  such  events  in  memory,  may  significantly  influence 
our  estimation  of  their  actual  likelihood. 

To  the  extent  that  availability  is  actually  associated  with  nonnative  frequency  or 
likelihood,  this  heuristic  can  be  a  useful  tool  for  intuitive  judgment  (Nisbett  &  Ross, 
1980,  p.  19).  After  all,  if  one  thing  actually  occurs  more  often  than  another  and  is 
therefore  more  probable,  we  can  probably  recall  more  instances  of  it.  And  events  that  are 
likely  to  occur  are  usually  easier  to  imagine  than  unlikely  events  (Heuer,  1999,  p.  148). 
For  example,  we  may  assess  the  risk  of  heart  attack  among  middle-aged  people  by 
recalling  such  occurrences  among  our  acquaintances.  Or  we  may  evaluate  the  probability 
that  a  given  business  venture  will  fail  by  imagining  the  various  difficulties  it  could 
encounter  (Kahneman  et  al,  1982,  p.  1 1). 
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Although  the  availability  heuristic  often  works  well,  people  are  frequently  led 
astray  when  the  ease  with  which  things  come  to  mind  is  influenced  by  factors  unrelated 
to  their  probability.  “The  ability  to  recall  instances  of  an  event  is  influenced  by  how 
recently  the  event  occurred,  whether  we  were  personally  involved,  whether  there  were 
vivid  and  memorable  details  associated  with  the  event,  and  how  important  it  seemed  at 
the  time”  (Heuer,  1999,  p.  148).  Yet  all  are  unrelated  to  the  event’s  true  probability. 

In  an  elementary  demonstration  of  this  effect,  Daniel  Kahneman  and  Amos 
Tversky  (1973)  presented  subjects  a  list  of  celebrities  and  subsequently  asked  them  to 
judge  whether  the  list  contained  more  men  or  more  women.  Different  lists  were 
presented  to  different  groups  of  subjects;  in  some,  the  men  were  more  famous,  and  in 
others,  the  women  were  more  famous.  For  each  of  the  lists,  the  subjects  consistently  and 
erroneously  concluded  that  the  gender  that  had  the  more  famous  celebrities  was  the  more 
numerous.  Kahneman  and  Tversky  (1973)  concluded  that  the  availability  bias  was  the 
source  of  the  error:  when  the  size  of  the  class  is  judged  by  the  availability  of  its  instances, 
a  class  whose  instances  are  easily  retrieved  will  appear  more  numerous  than  a  class  of 
equal  frequency  whose  instances  are  less  retrievable  (as  cited  in  Kahneman  et  al,  1982,  p. 
11). 

1.  Availability  and  Vividness 

“Information  may  be  described  as  vivid,  that  is,  as  likely  to  attract  and  hold  our 
attention  and  to  excite  our  imagination,  to  the  extent  that  it  is  emotionally  interesting, 
concrete  and  imagery-provoking,  [or]  proximate  in  a  sensory,  temporal,  or  spatial  way” 
(Nisbett  &  Ross,  1980,  p.  45).  Studies  have  shown  that  vivid  or  concrete  infonnation  has 
a  far  greater  impact  on  our  thinking  than  pallid  or  abstract  information,  even  when  the 
latter  may  have  greater  value  as  evidence.  For  example,  information  that  people  perceive 
directly,  that  we  hear  with  our  own  ears  or  see  with  our  own  eyes,  is  likely  to  be  more 
available  and  therefore  have  greater  impact  than  information  received  secondhand.  And 
case  histories  and  anecdotes  have  greater  impact  than  more  informative  but  abstract 
aggregate  or  statistical  data  (Heuer,  1999,  p.  1 16). 

This  tendency  to  assign  inferential  weight  to  information  in  proportion  to  its 
vividness  can  be  a  useful  heuristic:  “vivid  experiences  and  observations  can  be  a  source 
of  new  insights,  can  provide  phenomenological  reality  to  otherwise  poorly  understood 
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propositions,  and  can  inspire  action  in  circumstances  in  which  previous  knowledge  or 
opinion  has  not  overcome  inertia”  (Nisbett  &  Ross,  1980,  pp.  59-60).  Yet  because  the 
vividness  of  information  is  correlated  only  modestly  with  its  value  as  evidence,  there  are 
costs  associated  with  its  disproportionate  weighting. 

Richard  Nisbett  and  Eugene  Borgida  (1977)  conducted  a  series  of  experiments 
investigating  this  effect.  In  one,  introductory  psychology  students  were  asked  to  rate  the 
likelihood  that  they  would  enroll  in  a  number  of  upper-level  psychology  courses.  The 
students  then  divided  into  two  groups:  the  first  was  given  a  statistical  analysis  based  on 
evaluations  of  previous  students,  and  the  second  was  given  a  descriptive  version  of  the 
same  information  from  a  panel  of  previous  students.  Subjects  in  the  vivid  group  wanted 
to  take  more  of  the  highly  evaluated  courses  and  fewer  of  the  poorly  evaluated  courses, 
and  felt  more  confident  about  their  intentions,  than  did  their  counterparts  in  the  statistical 
group.  Nisbett  and  Borgida  (1977)  found  that  the  impact  of  vividness  was  most 
pronounced  for  those  students  who  intended  to  major  in  psychology  (for  whom  the 
course  selections  were  most  important  and  least  hypothetical).  The  two  concluded  that 
information  that  is  vivid,  concrete,  or  emotionally  pertinent,  or  of  temporal,  spatial,  or 
sensory  proximity  to  the  decision-maker,  has  far  greater  inferential  value  than  that  of 
abstract  or  statistical  information  of  equal  (or  even  greater)  value  (Nisbett  &  Ross,  1980, 
pp.  58). 

2.  Availability  and  Salience 

Salience  is  similar  to  vividness;  it  refers  to  the  fact  that  colorful,  dynamic,  or  other 
distinctive  stimuli  engage  people’s  attention  (Kahneman  et  al,  1982,  p.  192).  This 
manifestation  of  availability  can  become  an  undesirable  bias,  however,  when  such  stimuli 
disproportionately  engage  attention  and  accordingly  disproportionately  affect  judgment. 
Also  described  as  the  fundamental  attribution  error,  it  often  results  in  an  overestimation 
of  the  role  of  people  as  causal  agents  in  the  environment.  “That  is,  in  a  social  setting  in 
which  either  a  person  or  some  situational  variable  is  a  plausible  causal  candidate  for  an 
outcome,  there  exists  a  general  bias  to  see  people  as  causal  agents,  particularly  in  their 
enduring  dispositional  attributes”  (pp.  192-193). 

The  link  between  salience  and  causal  attribution  was  demonstrated  in  a  series  of 
experiments  conducted  by  Shelley  Taylor  and  Susan  Fiske  (1975).  In  one  of  the  more 
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famous,  the  two  had  six  observers  watch  a  two-man  conversation  from  one  of  three 
vantage  points:  seated  behind  one  of  the  men  who  was  talking,  seated  behind  the  other 
man  who  was  talking,  or  seated  on  the  sidelines  equidistant  from  the  two  men.  At  the 
end  of  the  conversation,  each  observer  was  asked  which  of  the  men  was  more  influential 
in  the  discussion.  Each  observer  with  a  clear  view  of  only  one  of  the  men  believed  that 
the  man  he  saw  clearly  had  set  the  tone  of  the  conversation,  had  determined  what 
information  would  be  exchanged,  and  had  caused  the  other  man  to  behave  as  he  did.  The 
observers  who  could  see  both  men  equally  well  felt  that  each  had  contributed  equally  to 
the  outcomes  (Pious,  1993,  p.  178).  Taylor  and  Fiske  (1975)  concluded  that  “the  more 
salient  a  potential  causal  agent  is,  the  greater  the  causal  role  observers  will  assign  to  that 
agent  in  accounting  for  particular  outcomes  of  events”  (as  cited  in  Nisbett  &  Ross,  1980, 
pp.  125). 

Shelley  Taylor  (1982)  notes  some  important  social  consequences  of  the  salience 
bias,  and  their  influence  on  judgment.  Examining  the  impact  of  solo  status  or  token 
integration  on  people’s  impressions  of  individuals,  Shelley  suggests: 

When  a  company  is  about  to  desegregate  and  include  members  of  a 
minority  group,  such  as  blacks,  women,  or  the  handicapped,  often  an 
intermediate  step  occurs  prior  to  full  integration.  In  this  step,  one  or  two 
members  of  this  previously  excluded  group  may  be  brought  into  what 
otherwise  has  been  a  white  male  work  group,  thus  creating  instances  of 
solo  status  (as  cited  in  Kahneman  et  al,  1982,  p.  193). 

Taylor  notes  that  the  significance  of  solo  status  is  its  novelty,  and  that  such 
distinctiveness  fosters  a  salience  bias.  And  although  her  example  may  be  somewhat 
outdated,  a  number  of  experiments  in  the  last  twenty  years  have  validated  her  conclusion 
that  when  an  individual  is  a  token,  a  solo  in  a  group,  or  is  in  some  other  way  salient,  they 
disproportionately  influence  the  judgment  of  observers. 
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IV.  ANCHORING 


The  concept  of  anchoring  was  first  introduced  by  Daniel  Kahneman  and  Amos 
Tversky  in  1974.  At  that  time,  it  was  seen  as  a  third  judgmental  heuristic,  akin  to 
representativeness  and  availability,  with  its  own  implications  for  systematic  error  in 
analysis  and  decision-making.  Today,  this  cognitive  process  is  understood  in  terms  of 
adjustment,  in  which  decision-makers  focus  on  an  anchor  and  then  make  a  series  of 
dynamic  adjustments  toward  a  final  estimate;  and  in  tenns  of  activation,  in  which  an 
anchor  acts  as  a  suggestion,  making  infonnation  consistent  with  the  anchor  more 
available.  In  both  cases,  when  employed  inappropriately,  such  anchoring  compounds  the 
effects  of  representativeness  and  availability  and  magnifies  the  degree  of  error  in 
judgment. 

A.  ANCHORING  AS  ADJUSTMENT 

In  a  variety  of  situations,  people  make  guesses  by  picking  some  natural  starting 
point  for  a  first  approximation  and  then  making  a  series  of  dynamic  adjustments  toward  a 
final  estimate.  Like  the  heuristics  described  in  the  previous  chapter,  such  anchoring  can 
be  helpful  in  making  judgments  and  decisions.  Consider  the  following  example: 

Imagine  that  you  are  trying  to  set  a  value  on  an  antique  chair  that  you  have 
inherited  from  a  distant  aunt.  You  might  recall  seeing  a  very  similar  chair 
in  slightly  better  condition  at  a  local  antique  dealer.  You  might  start  with 
that  price  as  an  anchor,  and  incorporate  the  difference  in  quality.  This 
seems  to  be  a  useful  and  effort-saving  use  of  anchoring  and  adjustment 
(Gilovich  et  al,  2002,  p.  120). 

Yet  in  many  cases,  people  fail  to  properly  adjust  from  the  anchor;  the  starting 
point  often  serves  as  a  drag  that  reduces  the  amount  of  adjustment,  so  the  final  estimate 
remains  closer  to  the  starting  point  than  it  ought  to  be  (Heuer,  1999,  p.  150).  Consider 
some  additional  infonnation  regarding  the  above  example: 

Now,  however,  imagine  that  you  had  seen  (on  Public  Television’s 
Antiques  Road  Show)  a  not-so-similar  chair  that,  unlike  yours,  is  signed 
by  the  designer  and  worth,  as  a  result,  many  of  thousands  of  dollars  more. 

If  you  were  to  use  this  as  an  anchor,  and  if  you  did  not  properly 
incorporate  the  fact  that  your  chair  did  not  have  a  signature,  you  might  end 
up  with  an  estimate  that  was  too  high  (Gilovich  et  al,  2002,  p.  120). 
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Under  the  influence  of  anchoring,  our  revisions  are  often  smaller  than  are  justified 
by  new  information,  and  our  final  estimates  are  often  biased  toward  the  initial  values 
(Schwenk,  1985,  p.  1 16).  It  is  not  only  people’s  eagerness  to  apply  simple  heuristics  like 
representativeness  and  availability  that  leads  to  problems;  it  is  also  the  failure  to  make 
necessary  adjustments  of  initial  judgments  (Nisbett  &  Ross,  1980,  p.  41).  In  many  cases, 
once  a  simple  heuristic  has  been  used,  subsequent  considerations  fail  to  exert  as  much 
impact  as  common  sense  or  normative  considerations  might  dictate  that  it  should  (p.  42). 

A  number  of  studies  have  shown  that  people  use  such  adjustment  to  solve  a 
variety  of  estimation  problems.  In  one  of  the  most  direct,  Amos  Tversky  and  Daniel 
Kahneman  (1974)  asked  subjects  to  adjust  an  arbitrary  initial  estimate  of  the  percentage 
of  African  countries  in  the  United  Nations.  Those  starting  with  anchors  of  10  percent 
produced  an  adjusted  estimate  of  25  percent;  those  with  anchors  of  65  percent  produced 
an  adjusted  estimate  of  45  percent.  Even  though  it  was  made  clear  that  the  anchors  were 
entirely  arbitrary  and  unrelated  to  the  judgment  task,  they  nevertheless  had  a  significant 
influence  on  final  estimates  (as  cited  in  Nisbett  &  Ross,  1980,  p.  41). 

B.  ANCHORING  AS  ACTIVATION 

In  recent  years,  researchers  have  suggested  that  the  origin  of  anchoring  lies  in  the 
influence  of  anchors  on  a  retrieval  stage  of  cognition,  rather  on  a  processing  stage.  In  this 
view,  an  anchor  acts  as  a  suggestion,  making  infonnation  consistent  with  the  anchor  more 
available  to  the  analyst  or  decision-maker,  either  in  memory  through  priming 
mechanisms  or  because  of  a  biased  external  search.  “Because  the  anchor  is  considered  as 
a  candidate  response  that  subjects  entertain,  at  least  as  a  transient  belief,  it  influences  the 
target  value”  (Gilovich  et  al,  2002,  p.  130). 

Fritz  Strack  and  Thomas  Mussweiler  (1997)  suggest  that  such  anchoring  is  a 
special  case  of  semantic  priming.  In  a  number  of  studies,  they  proposed  that  information 
retrieved  in  order  to  compare  an  anchor  to  a  target  is  consequently  more  available  for  use 
when  estimating  the  target  value  (as  cited  in  Gilovich  et  al,  2002,  p.  130).  Further,  they 
predicted  that  such  primed  information  influences  the  target  judgment  only  if  it  is 
relevant.  In  an  experiment  exploring  this  effect,  the  two  asked  subjects  to  estimate  the 
size  of  the  Brandenburg  Gate.  Strack  and  Mussweiler  (1997)  found  that  anchors 
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representing  the  width  of  the  Gate  had  a  significant  impact  on  judgments  of  its  width,  but 
little  influence  on  judgments  of  its  height. 

Karen  Jacowitz  and  Daniel  Kahneman  (1995)  argue  that  such  anchoring  occurs 
not  only  in  target  estimation,  but  also  in  comparative  judgment.  The  two  conducted  a 
number  of  studies  suggesting  that  the  retrieval  of  target  information  primed  by  an  anchor 
is  biased,  such  that  target  features  similar  to  the  anchor  are  retrieved  disproportionally  to 
that  which  is  normatively  appropriate  (Gilovich  et  al,  2002,  p.  131).  In  other  words,  the 
comparison  between  anchor  and  target  often  results  in  the  anchor  appearing  too  similar  to 
the  target.  Gretchen  Chapman  and  Eric  Johnson  (1999)  hypothesize  that  the  presence  of 
an  anchor  increases  the  availability  of  features  that  the  anchor  and  the  target  hold  in 
common  while  reducing  the  availability  of  features  of  the  target  that  differ  from  the 
anchor  (as  cited  in  Chapman  &  Johnson,  1999,  p.  131). 

Chapman  and  Johnson  (1999)  also  suggest  that  analysts  and  decision-makers 
concentrate  their  attention  on  features  of  the  target  that  are  similar  to  the  anchor.  In  an 
experiment  exploring  this  phenomenon,  the  two  had  subjects  compare  apartments 
described  by  three  attributes.  When  a  provided  anchor  value  was  high,  they  spent  more 
time  looking  at  positive  features  of  the  apartment.  When  the  anchor  value  was  low,  they 
spent  more  time  looking  at  negative  features  (Chapman  &  Johnson,  1999,  p.  131). 

In  a  similar  experiment,  Fritz  Strack  and  Thomas  Mussweiler  (1999)  had  subjects 
answer  an  anchoring  question  about  the  length  of  the  River  Elbe,  and  instructed  them  to 
list  the  features  of  the  target  that  came  most  easily  to  mind.  Subjects  that  were  given  a 
high  anchor  most  often  described  thoughts  that  implied  a  high  target  value,  whereas 
subjects  given  a  low  anchor  most  often  described  thoughts  that  implied  a  low  target  value 
(as  cited  in  Chapman  &  Johnson,  1999,  p.  13  l).io 


10  Strack  and  Mussweiler  (2000)  also  found  evidence  that  the  presence  of  an  anchor  primes  target 
features  that  are  similar  to  the  anchor.  In  that  study,  they  had  subjects  answer  the  anchoring  question,  “Is 
the  annual  mean  temperature  in  Germany  higher  or  lower  than  5  degrees  Celsius  (or  20  degrees  Celsius)?” 
and  then  participated  in  a  lexical  decision  task.  Subjects  given  the  low  anchor  were  faster  at  identifying 
words  such  as  “cold”  and  “snow,”  whereas  subjects  given  the  high  anchor  were  faster  at  identifying  words 
such  as  “hot”  and  “sun.”  The  two  concluded  that  the  anchor  primed  consistent  information  in  memory  (as 
cited  in  Chapman  &  Johnson,  1999,  p.  132). 
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C.  COGNITIVE  ANCHORING 

Richard  Nisbett  and  Lee  Ross  (1980)  describe  a  third  manifestation  of  anchoring 
that  involves  both  adjustment  and  activation.  The  two  suggest  that  once  analysts  or 
decision-makers  have  made  a  first  pass  at  a  problem,  their  initial  judgment  may  prove 
remarkably  resistant  to  further  information,  alternative  modes  of  reasoning,  or  even 
logical  or  evidential  challenges.  “Attempts  to  integrate  new  information  may  find  the 
individual  surprisingly  conservative,  that  is,  willing  to  yield  ground  only  grudgingly  and 
primed  to  challenge  the  relevance,  reliability,  or  authority  of  subsequent  information  or 
logical  consideration”  (Nisbett  &  Ross,  1980,  p.  42). 

People  hold  on  tightly  to  our  opinions  and  beliefs.  “We  are  confronted  almost 
daily  with  the  perversity  of  those  who  persist  in  their  misguided  political,  social,  and 
scientific  beliefs  even  after  we  have  infonned  them  of  the  facts”  (Nisbett  &  Ross,  1980, 
p.  167).  Theories  are  useful  because  they  structure  knowledge,  organize  experience,  and 
support  comparison  of  new  information  with  that  which  can  be  retrieved  from  memory. 
A  reluctance  to  hastily  abandon  our  theories  or  beliefs  is  often  appropriate;  neither 
laypeople  nor  scientists  should  reject  well-established  theory  simply  because  it  happens 
to  conflict  with  new  evidence.  “It  is  often  proper  to  look  askance  at,  or  even  to  totally 
ignore,  reports  of  virgin  births  or  new  cancer  cures.  And  it  may  even  be  proper  to 
dismiss  evidence  collected  by  reputable  scientists  if  it  conflicts  with  some  powerful, 
parsimonious,  and  integrative  theory”  (Nisbett  &  Ross,  1980,  p.  168). 

Yet  impressions  tend  to  persist  even  after  the  evidence  that  created  those 
impressions  has  been  fully  discredited  (Heuer,  1999,  p.  124).  Consider  the  following 
anecdotes: 

Jack  believes  that  he  dislikes  abalone  because  of  his  first  and  only 
exposure  to  that  food  and  then  is  assured  by  an  abalone  connoisseur  that 
he  sampled  an  inferior  frozen  product.  Jane  is  heartened  by  a 
correspondence  school’s  enthusiastic  appraisal  of  her  potential  as  a 
commercial  artist  and  then  finds  out  that  three  of  her  peers  who  answered 
the  same  advertisement  received  equally  glowing  assessments  (Nisbett  & 

Ross,  1980,  p.  175). 

Normatively,  it  is  clear  what  effect  discrediting  the  evidence  should  produce. 
Jack’s  opinion  on  abalone  and  Jane’s  belief  about  her  potential  as  an  artist  should  return 
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to  the  levels  they  were  at  before  hearing  the  (now-discredited)  evidence.  Studies  have 
shown,  however,  that  this  is  rarely  what  happens. 

Some  of  the  clearest  cases  of  the  persistence  of  impressions  based  on  discredited 
evidence  come  from  debriefing  experiments.  In  one  of  the  most  well-known,  Mark 
Lepper  and  Lee  Ross  (1975)  recruited  subjects  for  a  study  allegedly  concerned  with  the 
effects  of  performance  feedback,  in  which  they  were  asked  to  distinguish  between 
authentic  suicide  notes  and  fake  ones.  As  they  worked,  they  were  given  scripted 
feedback  that  suggested  how  well  they  had  performed.  One  group  was  told  they  had 
generally  succeeded,  and  the  other  group  was  told  that  they  had  generally  failed.  The 
subjects  in  both  groups  were  then  thoroughly  debriefed  about  the  predetermined  and 
random  nature  of  the  evaluation  of  their  performance.11  Yet  even  after  the  debriefing, 
subjects  in  the  “success”  condition  rated  their  performance  and  abilities  as  much  higher 
than  was  nonnatively  appropriate,  and  subjects  in  the  “failure”  condition  rated  theirs  as 
much  lower  than  was  nonnatively  appropriate  (as  cited  in  Heuer,  1999,  p.  124). 

This  experiment  and  others  have  led  to  a  number  of  important  conclusions 
regarding  such  cognitive  anchoring  and  the  perseverance  of  beliefs.  First,  when  people 
have  an  existing  theory,  exposure  to  probative  evidence  often  results  in  a  stronger  belief 
in  the  original  theory  than  is  nonnatively  appropriate.  Second,  when  people  approach 
evidence  without  an  existing  theory  and  then  form  a  theory  based  on  initial  evidence,  the 
theory  will  be  resistant  to  subsequent  evidence.  Finally,  when  people  formulate  a  theory 
based  on  some  putatively  probative  evidence  and  then  later  discover  that  the  evidence  is 
false,  the  theory  often  survives  (even  total)  discrediting  (Nisbett  &  Ross,  1980,  p.  169). 


11  The  subjects  were  not  only  told  that  their  feedback  had  been  false,  they  were  shown  the 
experimenter’s  instruction  sheet  assigning  them  to  the  success  or  failure  performance  condition  and 
specifying  the  feedback  to  be  presented  (Nisbett  &  Ross,  1980,  p.  177). 
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V.  PATTERNS  OF  ERROR  IN  THE  ANALYSIS  OF  IRAQI 
WEAPONS  OF  MASS  DESTRUCTION 


In  the  thirty  years  since  Kahneman  and  Tversky  (1972)  introduced  the  concept  of 
heuristics  and  biases,  researchers  have  conducted  countless  studies  and  experiments 
examining  human  judgment  under  uncertainty.  Yet  the  concept  is  more  than  academic; 
shortcomings  in  the  processes  that  underlie  judgment  can  have  the  most  serious  of 
consequences.  For  those  involved  in  national  security,  and  particularly  for  those  involved 
in  the  collection  and  analysis  of  national  intelligence,  perceptual  and  cognitive  bias  can 
be  quite  literally  a  matter  of  life  and  death.  The  judgment  made  about  Iraqi  weapons  of 
mass  destruction  represents  just  such  a  case. 

In  the  spring  and  summer  of  2002,  as  the  Bush  Administration  contemplated  an 
invasion  of  Iraq,  the  Intelligence  Community12  was  charged  with  a  comprehensive 
assessment  of  Saddam  Hussein’s  nuclear,  biological,  and  chemical  weapons  programs 
and  ambitions.  This  assessment  was  codified  in  a  National  Intelligence  Estimate  (NIE)  in 
October  2002.  Published  by  the  National  Intelligence  Council13  and  entitled  Iraq’s 
Continuing  Programs  for  Weapons  of  Mass  Destruction,  the  NIE  assessed  that  Iraq  had 
continued  its  weapons  of  mass  destruction  programs  in  defiance  of  U.N.  resolutions  and 
restrictions.  Specifically,  it  concluded  that  Iraq  had  reconstituted  its  nuclear  weapons 
program  and  could  construct  a  device  by  the  end  of  the  decade;  that  Iraq’s  biological 
weapons  program  was  larger  and  more  advanced  than  it  was  before  the  Gulf  War;  that 
Iraq  had  begun  renewed  production  of  chemical  warfare  agents  and  had  a  few  hundred 
metric  tons  in  stockpiles;  and  that  Iraq  was  developing  delivery  systems  for  its  nuclear, 


12  The  Intelligence  Community  is  a  federation  of  executive  branch  agencies  and  organizations  that 
conduct  intelligence  activities  necessary  for  conduct  of  foreign  relations  and  protection  of  national  security. 
It  is  comprised  of  15  Federal  government  agencies,  services,  bureaus,  or  other  organizations  within  the 
executive  branch  that  play  a  role  in  national  intelligence,  including  the  Army,  Navy,  Air  Force,  and  Marine 
Corps  Intelligence  Elements;  the  Central  Intelligence  Agency;  the  Defense  Intelligence  Agency;  the 
Department  of  Homeland  Security;  the  Energy  Department;  the  Federal  Bureau  of  Investigation;  the 
National  Geospatial-Intelligence  Agency;  the  National  Reconnaissance  Office;  the  National  Security 
Agency,  the  State  Department;  the  Treasury  Department;  and  the  United  States  Coast  Guard 
(http://www.intelligence.gov/ic_in_a_nutshell/index.htm,  n.d.). 

13  The  National  Intelligence  Center  (NIC)  is  a  center  of  strategic  thinking  within  the  US  Government, 
reporting  to  the  Director  of  National  Intelligence  (DNI)  and  providing  the  President  and  senior 
policymakers  with  analyses  of  foreign  policy  issues  that  have  been  reviewed  and  coordinated  throughout 
the  Intelligence  Community  (http://www.cia.gov/nic/NIC_home.html,  n.d.). 
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biological,  and  chemical  weapons  that  could  be  employed  against  deployed  forces  and 
potentially  against  the  continental  United  States  (CIA,  2002). 

Yet  each  one  of  these  assessments,  which  together  served  as  the  basis  for  the 
Bush  Administration’s  decision  to  go  to  war,  was  ultimately  proven  wrong.  Extensive 
investigations  by  the  Iraq  Survey  Group  (ISG)  after  the  war  found: 

No  evidence  that  Iraq  had  tried  to  reconstitute  its  capability  to  produce 
nuclear  weapons  after  1991;  no  evidence  of  biological  warfare  (BW)  agent 
stockpiles  or  of  mobile  biological  weapons  production  facilities;  and  no 
substantial  chemical  warfare  (CW)  stockpiles  or  credible  indications  that 
Baghdad  had  resumed  production  of  CW  after  1991  (Silberman  &  Robb, 

2005,  p.  45). 

How  could  the  Intelligence  Community  have  gotten  it  so  wrong?  The  errors  were 
not  the  result  of  simple  bad  luck,  or  a  once-in-a-lifetime  “perfect  storm,”  as  some  have 
suggested.  The  Senate  Intelligence  Committee,  chaired  by  Pat  Roberts  and  John 
Rockefeller,  and  the  Commission  on  the  Intelligence  Capabilities  of  the  United  States 
Regarding  Weapons  of  Mass  Destruction,  chaired  by  Lawrence  Silbennan  and  Charles 
Robb,  both  concluded  that  these  errors  were  instead  the  product  of  poor  intelligence 
collection,  an  analytical  process  that  was  driven  by  assumptions  and  inferences  rather 
than  data,  and  inadequate  validation  and  vetting  of  dubious  intelligence  sources 
(Silberman  &  Robb,  2005,  p.  47).  Yet  despite  tens  of  thousands  of  hours  of  interviews 
and  testimony,  and  thousands  of  pages  of  published  reports,  neither  the  Senate 
Intelligence  Committee  nor  the  WMD  Commission  acknowledged  the  significant  role  of 
perceptual  and  cognitive  bias  in  the  errors  in  judgment  that  led  to  war. 

A.  THE  IMPACT  OF  BIAS  ON  PERCEPTION  AND  JUDGMENT 
1.  Representativeness  in  the  Analysis  of  Aluminum  Tubes 
By  October  2002,  the  Intelligence  Community  had  concluded  that  more  than  ten 
years  of  sanctions  and  the  loss  of  much  of  Iraq’s  physical  nuclear  infrastructure  under 
IAEA  oversight  had  not  diminished  Saddam’s  interest  in  nuclear  weapons.  And  though 
there  were  a  number  of  factors  that  contributed  to  this  assessment,  it  hinged  on  two  key 
judgments.  First,  the  Intelligence  Community  saw  Iraq’s  efforts  to  procure  tens  of 
thousands  of  proscribed  high-strength  aluminum  tubes  as  clear  evidence  of  a  fully 
reconstituted  nuclear  weapons  program,  noting,  “All  intelligence  experts  agree  that  Iraq 
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is  seeking  nuclear  weapons  and  that  these  tubes  could  be  used  in  a  centrifuge  enrichment 
program”  (CIA,  2002).  Second,  the  Intelligence  Community  concluded  that  although  it 
was  unlikely  that  Iraq  could  indigenously  produce  enough  weapons-grade  material  for  a 
deliverable  nuclear  weapon  until  the  last  half  of  this  decade,  Baghdad  could  produce  a 
nuclear  weapon  within  a  year  if  it  were  able  to  procure  weapons-grade  fissile  material 
abroad  (CIA,  2002).  Both  judgments  were  largely  the  result  of  errors  due  to 
representativeness. 

As  Chapter  III  describes,  representativeness  is  one  of  the  most  basic  and  most 
often  employed  heuristics  that  people  use  to  make  judgments.  It  allows  people  to  reduce 
otherwise  taxing  inferential  tasks  into  simple  similarity  judgments,  in  which  objects, 
events,  or  processes  are  assigned  to  one  conceptual  category  rather  than  to  another  based 
on  how  well  they  represent  or  resemble  one  category  more  than  another  (Nisbett  &  Ross, 
1980,  p.  7).  Problems  with  representativeness  arise  when  we  use  it  as  our  primary  (or  our 
only)  judgmental  strategy;  the  otherwise  useful  heuristic  becomes  an  undesirable  bias 
when  we  overestimate  the  extent  to  which  a  situation  or  sample  is  representative  of  the 
situation  or  population  to  which  we  wish  to  generalize  (Schwenk,  1985,  p.  121). 

The  Intelligence  Community  insisted  that  Iraq’s  efforts  to  procure  more  than 
60,000  high-strength  aluminum  tubes14  proved  Baghdad’s  intentions  to  reconstitute  its 
nuclear  weapons  programs  because  the  dimensions  of  the  tubes  were  similar  to  the 
“Beams”  configuration,  a  design  used  by  Iraq  before  the  Gulf  War;  and  nearly  matched 
the  “Zippe”  configuration,  another  publicly  available  design  for  gas  centrifuges  (Senate 
Intelligence  Committee,  2004,  p.  89). 15  In  the  twelve  months  following  this  original 
assessment,  CIA  produced  at  least  nine  additional  reports  discussing  the  aluminum  tube 


14  In  2001,  the  CIA  learned  that  the  tubes  were  to  be  made  of  a  specific  grade  of  aluminum  alloy 
which,  due  to  its  hardness  and  strength,  and  resulting  suitability  for  nuclear  applications,  was  prohibited  for 
use  in  Iraq  by  a  number  of  United  Nations  Security  Council  Resolutions.  The  tubes  were  to  be 
manufactured  from  7075-T6  aluminum,  with  an  outer  diameter  of  81mm,  and  an  inner  diameter  of  74.4mm, 
a  wall  thickness  of  3.3mm,  and  a  length  of  900mm.  The  tubes  were  to  be  anodized  using  chromic  acid  and 
were  to  be  shipped  wrapped  in  wax  paper  and  separated  from  each  other.  This  type  of  aluminum  alloy  is 
extremely  hard  and  strong  when  formed  into  a  tube  of  more  than  75mm  in  diameter.  Moreover,  it  is  a 
controlled  item  under  the  Nuclear  Suppliers  Group  and  Annex  III  of  U.N.  Security  Council  Resolution  687 
and  707;  these  resolutions  prohibited  Iraq  from  importing  the  material  because  it  could  have  nuclear 
applications  (Senate  Intelligence  Committee,  2004,  p.  88). 

15  The  CIA,  and  later  the  National  Ground  Intelligence  Center  (NGIC),  assessed  that  the  tubes  “had 
little  use  other  than  for  a  uranium  enrichment  program”  (Senate  Intelligence  Committee,  2004,  p.  88). 
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procurement  efforts.  Yet  none  provided  any  additional  information  to  support  the  CIA’s 
analysis  that  the  tubes  were  intended  for  Iraq’s  nuclear  program  (p.  90).  The  complex 
inferential  task  of  determining  technical  suitability  was  reduced  into  a  simple  judgment  of 
similarity.  Just  as  “people  express  great  confidence  in  the  prediction  that  a  person  is  a 
librarian  when  a  given  description  of  his  personality  which  matches  the  stereotype  of 
librarians,  even  if  the  description  is  scanty,  unreliable,  or  outdated”  (Kahneman  et  al, 
1982,  p.  9),  the  CIA  expressed  great  confidence  that  the  tubes  were  for  uranium 
enrichment  because  their  description  matched  the  Zippe  design. 

This  judgment  about  the  aluminum  tubes  was  also  based  on  an  illusion  of  validity. 
In  examining  whether  the  walls  of  the  tubes  were  of  the  necessary  thickness  for 
centrifuge  use,  the  CIA  sought  the  assistance  of  an  independent  contractor  to  perform  a 
number  of  tests.  The  agency  provided  the  contractor  with  stacks  of  intelligence  data  and 
analysis  on  Iraqi  tube  procurements,  and  the  National  Ground  Intelligence  Center’s 
analysis  that  the  tubes  were  most  likely  for  use  in  uranium  enrichment.! 6  After  an  initial 
test  questioned  the  suitability  of  the  tubes  for  use  in  enrichment,  the  CIA  complained  that 
methodology  used  by  the  contractor  was  faulty  (Silberman  &  Robb,  2005,  p.  70).  The 
contractors  eventually  provided  a  “correction”  with  new  test  data,  which,  the  CIA 
believed,  demonstrated  that  the  tubes  had  sufficient  strength  for  use  in  a  centrifuge.  Their 
report  concluded  that  “the  tubes  [were]  consistent  with  design  requirements  of  gas 
centrifuge  rotors”  and  “inconsistent  with  [conventional]  rocket  motor  applications” 
(Senate  Intelligence  Committee,  2004,  p.  94).  Although  there  was  no  evidence  of 
impropriety,  the  testing  is  an  example  of  how  unwarranted  confidence,  spawned  by  an 
apparent  fit  between  the  input  information  and  the  predicted  outcome,  produced  an 
illusion  of  validity. 

2.  Representativeness  in  the  Analysis  of  Samarra  Tanker  Trucks 

By  October  2002,  the  Intelligence  Community  had  also  described  its  “high 
confidence”  in  the  fact  that  Iraq  possessed  chemical  weapons  and  had  begun  to 
reconstitute  its  chemical  weapons  program.  Specifically,  analysts  believed  that  gaps  in 
Iraqi  accounting  and  an  analysis  of  current  production  capabilities  strongly  suggested  that 

16  Although  a  number  of  analysts  from  the  Department  of  Energy  (DOE)  disagreed  with  the 
assessment  that  the  tubes  were  for  use  in  uranium  enrichment,  their  reports  were  never  given  to  the 
independent  contractor  (Senate  Intelligence  Committee,  2004,  p.  94). 
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Iraq  had  a  stockpile  of  VX,  sarin,  cyclosarin,  and  mustard  (CIA,  2002).  And  although  the 
NIE  cautioned  that  the  Intelligence  Community  had  “little  specific  information  on  Iraq’s 
CW  stockpile,”  it  estimated  that  “Saddam  probably  [had]  stocked  at  least  100  metric  tons 
and  possibly  as  much  as  500  metric  tons  of  CW  agents  (Silberman  &  Robb,  2005,  p. 
112).  The  Intelligence  Community  further  concluded  that  “much”  of  Iraq’s  CW 
stockpiles  had  been  produced  in  the  past  year,  that  Iraq  had  “rebuilt  key  portions  of  its 
CW  infrastructure,”  and  that  Baghdad  planned  to  expand  dual-use  infrastructure  that  it 
could  divert  quickly  to  CW  production  (CIA,  2002).  This  judgment,  based  substantially 
on  the  presence  of  Samarra  tanker  trucks  at  key  ammunition  depots  and  production 
facilities,  was  also  influenced  by  bias. 

Chapter  III  discusses  how  representativeness  can  also  manifest  itself  in  our 
perception  of  cause  and  effect.  Because  we  do  not  have  an  intuitive  understanding  of  the 
kinds  and  amount  of  information  needed  to  prove  a  relationship,  people  often  perceive 
relationships  that  do  not  in  fact  exist  (Heuer,  1999,  p.  127).  Richard  Nisbett  and  Thomas 
Wilson  (1977)  suggest  that  “people  have  strong  a  priori  notions  of  the  types  and  causes 
that  ought  to  be  linked  to  particular  types  of  effects,  and  a  simple  ‘resemblance  criterion’ 
often  figures  heavily  in  such  notions”  (as  cited  in  Nisbett  &  Ross,  1980,  p.  115). 

The  “Samarra”  tanker  is  a  specific  type  of  delivery  truck  that  was  known  to  have 
been  used  by  Iraq  for  chemical  weapons  shipments  throughout  the  1980s  and  before  the 
Gulf  War  (Silbennan  &  Robb,  2005,  p.  122).  In  the  spring  and  summer  of  2002,  satellite 
imagery  suggested  that  a  large  number  of  these  trucks  were  transporting  materials  to  and 
from  chemical  plants  and  ammunition  depots.  Because  of  the  similarity  of  this  platform 
and  activity  with  the  trucks  and  shipment  patterns  known  to  be  involved  in  chemical 
weapons  programs  in  the  1980s,  Community  analysts  concluded  that  current  imagery 
confirmed  chemical  weapons  activity.  Instead  of  the  detailed  and  multifaceted  analysis 
that  this  determination  required,  the  Intelligence  Community  again  defaulted  to  a  simple 
similarity  judgment.  Analysts  then  compounded  this  error  in  their  perception  of 
relationships  that  did  not  exist,  as  shown  in  a  series  of  faulty  cause  and  effect 
determinations: 

Based  on  the  assessment  that  the  presence  of  these  Samarra-type  trucks 

suggested  CW  shipments,  CW  analysts  then  judged  that  the  frequency  of 
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such  transshipments  pointed  to  the  assessment  that  “CW  was  already 
deployed  with  the  military  logistics  chain,”  which,  in  turn,  indicated  to 
these  analysts  that  Iraq  had  added  to  its  CW  stockpile  in  the  last  year. 

That  assessment,  in  turn,  indicated  to  analysts  that  Iraq  had  restarted  CW 
production  (Silberman  &  Robb,  2005,  p.  123). 

As  this  logic  train  illustrates,  the  October  2002  NIE  conclusion  that  Iraq  had 
reconstituted  its  chemical  weapons  programs  was  fundamentally  based  on  the  biased 
assessment  that  the  “Samarra”  trucks  seen  by  imagery  satellites  were  in  any  way  involved 
in  chemical  weapon  activity. 

3.  Availability  in  the  Analysis  of  Uranium  from  Niger 

The  Intelligence  Community’s  October  2002  conclusion  that  Iraq  was 
reconstituting  its  nuclear  weapons  program  was  punctuated  by  the  assessment  that  “Iraq 
was  ‘trying  to  procure  uranium  ore  and  yellowcake’17  and  that  ‘a  foreign  government 
service’  had  reported  that  ‘Niger  planned  to  send  several  tons’  of  yellowcake  to  Iraq” 
(Silberman  &  Robb,  2005,  p.  76).  Based  almost  entirely  on  a  handful  of  reports  from  a 
foreign  intelligence  service,  this  judgment  was  due,  at  least  in  part,  to  the  availability 
bias. 

As  Chapter  III  describes,  our  judgment  about  the  likelihood  of  an  event  is  often 
based  on  its  availability  in  the  processes  of  perception,  memory,  or  construction  from 
imagination  (Nisbett  &  Ross,  1980,  p.  18).  Cues  such  as  the  ease  with  which  a  scenario 
or  event  comes  to  mind,  or  the  frequency  of  such  events  in  memory,  exert  a  powerful 
influence  on  our  estimation  of  their  actual  likelihood.  Problems  result  when  these  cues 
are  not  normatively  appropriate,  or  when  we  associate  the  likelihood  of  a  scenario  or 
event  with  infonnation  that  is  vivid,  concrete,  or  emotionally  pertinent,  or  of  temporal, 
spatial,  or  sensory  proximity  to  the  decision-maker,  when  these  factors  are  completey 
unrelated  to  its  true  probability. 

From  the  time  that  the  reports  of  uranium  from  Niger  first  surfaced  in  2001,  the 
CIA  recognized  the  potential  impact  with  policymakers.  In  a  cable  requesting  additional 
information  from  the  foreign  intelligence  service  that  first  made  the  claims,  the  CIA 

17  Yellowcake  (also  known  as  urania  and  uranic  oxide)  is  concentrated  uranium  oxide,  obtained 
through  the  milling  of  uranium  ore.  It  is  created  by  passing  raw  uranium  ore  through  crushers  and  grinders 
to  produce  “pulped”  ore.  This  is  then  bathed  in  sulfuric  acid  to  leach  out  the  uranium.  Y ellowcake  is  what 
remains  after  drying  and  filtering  (Retrieved  November  20,  2005  from  http://www.wikipedia.com). 
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acknowledged  that  “the  issue  of  Iraqi  uranium  procurement  continues  to  resonate  with 
senior  policymakers”  (Senate  Intelligence  Committee,  2004,  p.  67).  And  analysts  within 
the  Department  of  Energy  cautioned  their  counterparts  within  the  CIA  against 
“converting  strong  statements  [about  uranium  from  Niger]  into  the  knockout  punch”  (p. 
60). 18  Yet  the  terms  “uranium”  and  “yellowcake”  seemed  to  be  as  radioactive  as  the 
elements  themselves.  And  because  the  vividness  of  this  information  was  correlated  only 
modestly  with  its  value  as  evidence,  the  Intelligence  Community  paid  a  high  price  in 
credibility  for  its  disproportionate  weighting. 

4.  Availability  in  the  Analysis  of  Mobile  Biological  Weapons  Labs 

The  October  2002  NIE  also  summarized  the  conclusions  of  the  Intelligence 
Community  regarding  Iraq’s  biological  warfare  capabilities  and  ambitions,  noting,  “All 
key  aspects,  R&D,  production,  and  weaponization,  of  Iraq’s  offensive  BW  program  are 
active  and  most  elements  are  larger  and  more  advanced  than  they  were  before  the  Gulf 
war”  (CIA,  2002).  In  particular,  the  Community  believed  that  Iraq  had  some  lethal  and 
incapacitating  BW  agents  and  was  capable  of  quickly  producing  and  weaponizing  a 
variety  of  such  agents,  including  anthrax,  for  delivery  by  bombs,  missiles,  aerial  sprayers, 
and  covert  operatives,  potentially  against  the  U.S.  Homeland  (CIA,  2002).  Further,  the 
Community  believed  that  Baghdad  had  established  a  large-scale,  redundant,  and 
concealed  biological  agent  production  capability,  which  included  mobile  facilities  that 
could  evade  detection,  were  highly  survivable,  and  could  exceed  the  production  rates  Iraq 
had  prior  to  the  Gulf  war  (Silberman  &  Robb,  2005,  p.  80).  This  conclusion  too  was 
influenced  by  errors  in  judgment  due  to  availability. 

Chapter  III  describes  salience  as  similar  to  vividness  in  that  it  refers  to  the  fact 
that  colorful,  dynamic,  or  other  distinctive  stimuli  engage  people’s  attention  (Kahneman 
et  al,  1982,  p.  192).  Systematic  errors  in  judgment  occur,  however,  when  our  attention  is 
engaged  disproportionately  compared  with  what  is  normatively  appropriate.  Moreover, 
people,  particularly  when  they  have  token  or  solo  status,  are  often  seen  as  particularly 


18  The  IAEA  later  found  that  the  documents  supporting  the  claims  of  uranium  from  Niger  contained 
numerous  indications  of  forgery,  including  flaws  in  the  letterhead,  forged  signatures,  misspelled  words, 
incorrect  titles  for  individuals  and  government  entities,  and  anomalies  in  the  documents’  stamps,  as  well  as 
serious  errors  in  content  (Silberman  &  Robb,  2005,  p.  77). 


39 


salient.  As  Shelley  Taylor  (1982)  suggests,  such  people  often  disproportionately 
influence  the  judgment  of  observers  (as  cited  in  Kahneman  et  al,  1982,  p.  193). 

The  detenninations  regarding  the  reconstitution  of  Iraq’s  biological  weapons 
programs  were  based  almost  exclusively  on  reporting  from  a  single  human  source, 
codenamed  “Curveball,”  who  insisted  that  Iraq  had  mobile  facilities  for  producing  such 
agents  (Silberman  &  Robb,  2005,  p.  80)99  There  is  no  doubt  that  “Curveball”  was 
appealing  as  a  source;  a  number  of  factors  made  him  particularly  salient.  First,  his 
position  as  an  Iraqi  defector  made  him  fairly  unique,  and  granted  him  the  solo  status  that 
Taylor  describes  in  her  study.  Second,  his  status  as  a  chemical  engineer,  coupled  with  his 
access  to  specific  production  facilities  within  Iraq,  resulted  in  an  uncommon  level  of 
technical  detail  and  accuracy  and  gave  him  a  great  deal  of  credibility  with  his  handlers. 
Finally,  his  hundreds  of  compelling  eye-witness  accounts  seemed  to  repeatedly  trump 
more  abstract  indicators  that  contradicted  his  assertions.  In  the  end,  the  analysts’ 
resistance  to  any  infonnation  that  could  undermine  Curveball ’s  reliability  suggests  that 
the  analysts  were  “unduly  wedded”  to  a  highly  salient  source  that  fully  supported  their 
expectations  and  assumptions  (Silbennan  &  Robb,  2005,  p.  90). 

B.  THE  IMPACT  OF  ANCHORING  ON  PERCEPTION  AND  JUDGMENT 

Many  within  the  Intelligence  Community  have  acknowledged  that  their  starting 
point  for  evaluating  Iraq’s  WMD  programs  before  the  2003  invasion  was  Iraq’s  past. 
Assumptions  were  formed  based  on  Iraq’s  history  of  producing  biological  and  chemical 
weapons,  its  use  of  chemical  weapons,  its  history  of  effectively  concealing  its  nuclear 
program  before  the  Gulf  War,  and  the  regime’s  failure  to  account  for  its  previously 
declared  stockpiles.  “Analysts  operated  from  the  premise  that  Iraq  very  likely  still 
possessed  biological  and  chemical  weapons,  was  still  hiding  [them]  from  inspectors,  and 
was  still  seeking  to  rebuild  its  nuclear  weapons  program”  (Silberman  &  Robb,  2005,  p. 
168).  The  analytical  flaw  was  not  that  this  premise  was  unreasonable  (for  it  was  not); 
rather,  it  was  that  this  premise  hardened  into  a  presumption,  and  analysts  began  to  fit  the 


19  In  fact,  it  was  almost  entirely  due  to  “Curveball”  that  the  Intelligence  Community  changed  its 
assessment  of  the  1990s  that  Iraq  could  have  biological  weapons  to  the  assessment  of  the  October  2002 
NIE  that  Iraq  had  biological  weapons  (Silberman  &  Robb,  2005,  p.  84). 
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facts  to  the  theory  rather  than  the  other  way  around.  The  premise  became  an  anchor  that 
compounded  the  effects  of  representativeness  and  availability  and  magnified  the  degree 
of  error  in  judgment. 

1.  Anchoring  and  the  Influence  of  Expectations 

The  experiment  by  Jerome  Bruner  and  Leo  Postman  (1949),  during  which  playing 
cards  were  altered  so  that  the  spades  were  red  and  the  hearts  black,  demonstrates  the 
impact  of  expectations  on  human  perception.  It  shows  how  people  develop  expectations 
from  a  myriad  of  diverse  sources,  including  education,  organizational  norms,  cultural 
values,  and  role  requirements,  as  well  as  by  the  stimuli  recorded  by  our  receptor  organs. 
In  particular,  the  experiment  shows  how  perceptual  organization  is  powerfully 
determined  by  expectations  built  upon  past  experience  (as  cited  in  Chandler,  2004). 

This  was  certainly  the  case  with  the  U.S.  Intelligence  Community  in  its 
assessment  of  Iraqi  WMD;  its  collective  experience  with  Baghdad  throughout  the  two 
decades  preceding  the  invasion  anchored  its  pre-war  assessment  and  limited  its  analytical 
adjustment.  In  particular,  a  number  of  events  led  to  powerful  and  well-defined 
expectations  about  Iraq’s  ambitions  regarding  WMD. 

First,  there  was  no  doubt  that  Saddam  developed  and  maintained  huge  stockpiles 
of  chemical  weapons  throughout  the  1980s.  Following  the  Gulf  War,  the  U.N.  Special 
Commission  (UNSCOM)  supervised  the  destruction  of  more  than  40,000  of  these 
chemical  munitions,  nearly  500,000  liters  of  chemical  agents,  1.8  million  liters  of 
chemical  precursors,  and  seven  different  types  of  delivery  systems,  including  ballistic 
missile  warheads  (CIA,  2002). 

Second,  there  was  no  dispute  that  Saddam  had  used  chemical  weapons  during  his 
war  with  Iran.  In  at  least  ten  different  large  scale  attacks  over  seven  years,  Iraqi  forces 
killed  or  injured  more  than  20,000  people,  delivering  chemical  agents,  including  mustard 
agent  and  the  nerve  agents  sarin  and  tabun2,  in  aerial  bombs,  122mm  rockets,  and 
artillery  shells  (CIA,  2002).  On  March  16,  1988,  Saddam  used  chemical  weapons  against 
his  own  people  when  Iraqi  warplanes  bombed  the  town  of  Halabja,  where  80,000  Kurds 
lived.  As  many  as  5,000  people  are  said  to  have  died,  according  to  international  human 
rights  organizations  (CIA,  2002). 
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Third,  during  inspections  in  Iraq  after  the  Gulf  War,  UNSCOM  was  shocked  by 
the  degree  to  which  they  underestimated  Iraq’s  nuclear  weapons  program.  According  to 
David  Kay,  U.N.  Chief  Weapons  Inspector  from  1983  through  1992,  inspectors  were 
particularly  surprised  by  the  magnitude  and  advanced  state  of  Iraq’s  efforts  to  obtain 
nuclear  explosive  devices: 

At  the  time  of  the  Gulf  War,  Iraq  was  probably  only  18  to  24  months  away 
from  its  first  crude  nuclear  device  and  no  more  than  three  to  four  years 
away  from  more  advanced,  deliverable  weapons.  Moreover,  the  amount 
of  foreign  assistance  and  technology  that  had  fueled  the  Iraqi  arms 
program  was  truly  staggering  (Kay,  1995). 

Kay  believed  that  the  failure  of  both  International  Atomic  Energy  Agency  (IAEA) 
safeguards  inspectors  and  national  intelligence  authorities  to  detect  a  nuclear  weapons 
program  of  the  magnitude  and  advanced  character  of  Iraq’s  “should  stand  as  a  monument 
to  the  fallibility  of  on-site  inspections  and  national  intelligence  when  faced  by  a 
determined  opponent”  (Kay,  1995).  It  did;  as  the  October  2002  NIE  suggests,  this 
monument  enshrined  the  expectations  that  Iraq  was  behaving  as  it  had  in  the  past,  and 
that  the  Intelligence  Community  had  to  be  careful  to  avoid  being  caught  off-guard.20 

Finally,  even  as  late  as  2002,  Iraq  had  never  fully  accounted  for  major  gaps  and 
inconsistencies  in  its  declarations  and  has  provided  no  credible  proof  that  it  has 
completely  destroyed  its  weapons  stockpiles  and  production  infrastructure  (CIA,  2002). 
According  to  Kenneth  Pollack,  fonner  CIA  analyst  and  Director  of  Persian  Gulf  Affairs 
for  the  National  Security  Council,  the  discrepancies  between  how  much  WMD  material 
went  into  Iraq  and  how  much  Iraq  could  prove  it  had  destroyed  became  the  final  element 
in  the  context  for  the  Intelligence  Community’s  pre-war  assessment.  “The  U.N. 
inspectors  obtained  virtually  all  the  import  figures.  They  then  asked  the  Iraqis  to  either 
produce  the  materials  or  account  for  their  destruction.  In  many  cases  the  Iraqis  could 
not”  (Pollack,  2004). 

The  Intelligence  Community’s  assessment  of  Iraq’s  pre-war  WMD  programs  was 
clearly  not  made  in  a  vacuum.  Rather,  as  the  Intelligence  Community  later  explained,  its 
assessments  “were  informed  by  its  analysis  of  Iraq’s  ambitions  and  capabilities  spanning 

20  The  NIE  specifically  took  note  of  “revelations  after  the  Gulf  war  [that]  starkly  demonstrated  the 
extensive  efforts  undertaken  by  Iraq  to  deny  information”  (CIA,  2002). 
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the  preceding  fifteen  years,  as  well  as  by  lessons  learned  from  over  a  decade  of  dealing 
with  Iraqi  intransigence  on  this  issue”  (Silbennan  &  Robb,  2005,  p.  53).  Given  Saddam’s 
history  of  using  them,  his  previous  deceptions,  and  his  repeated  efforts  to  obstruct  the 
U.N.,  many  people  still  wonder  how  the  Intelligence  Community  could  have  concluded 
that  Saddam  Hussein  did  not  have  weapons  of  mass  destruction  (Silbennan  &  Robb, 
2005,  p.  46). 

2.  Anchoring  and  the  Influence  of  Situational  Context 

The  experiment  conducted  by  B.R.  Bugelski  and  Delia  Alampay  (1961),  in  which 
subjects  evaluated  a  drawing  that  could  be  interpreted  either  as  rat  or  an  old  man  wearing 
spectacles,  demonstrates  how  different  circumstances  evoke  different  sets  of  expectations 
(as  cited  in  Chandler,  2004).  In  the  forty  years  since  then,  countless  studies  have 
validated  the  conclusion  that  we  perceive  objects  or  events  differently  based  on  the 
situation.  In  other  words,  our  perception  is  influenced  by  the  context  in  which  it  occurs. 

Just  as  its  past  experience  with  Baghdad  served  as  an  anchor  that  influenced  the 
Intelligence  Community’s  perceptions,  so  too  did  the  current  situation.  In  fact,  a  number 
of  key  events  occurred  just  prior  to  the  decision  to  invade  Iraq  that  resulted  in  a  context 
that  seemed  to  support  many  of  the  pre-war  assessments. 

First,  there  was  no  doubt  that  Saddam  was  a  brutal  dictator  who  ruled  Iraq 
through  a  combination  of  violence,  secrecy,  mendacity,  and  fear  (Silbennan  &  Robb, 
2005,  p.  147).  Up  until  the  very  end,  his  regime  operated  a  robust  and  ruthless  security 
system  and  engaged  in  sophisticated  efforts  to  conceal  or  disguise  its  activities  from 
outside  intelligence  services.  The  Intelligence  Community  characterized  these  efforts  as 
“denial  and  deception,”  and  pointed  to  them  as  evidence  that  Saddam  was  clearly  hiding 
something  (p.  48). 

Second,  there  was  Iraqi  rhetoric.  Consider  what  intelligence  analysts  must  have 
thought  after  hearing  Saddam  make  the  following  statement  in  June  of  2000: 

If  the  world  tells  us  to  abandon  all  our  weapons  and  keep  only  swords,  we 
will  do  that ...  if  they  destroy  their  weapons.  But  if  they  keep  a  rifle  and 
then  tell  me  that  I  have  the  right  to  possess  only  a  sword,  then  we  would 
say  no.  As  long  as  the  rifle  has  become  a  means  to  defend  our  country 
against  anybody  who  may  have  designs  against  it,  then  we  will  try  our  best 
to  acquire  the  rifle  (Pollack,  2005). 
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Kenneth  Pollack  (2005)  notes  that  in  this  context,  it  would  be  very  difficult  not  to 
interpret  Saddam’s  remarks  as  an  announcement  that  he  intended  to  reconstitute  his 
WMD  programs. 

Finally,  there  was  Iraq’s  failure  to  comply  with  U.N.  Resolution  1441,  which 
declared  Iraq  in  material  breach  of  its  obligations  under  previous  resolutions,  required 
new  weapons  declarations  from  Iraq,  and  included  stringent  provisions  for  Iraqi 
compliance,  including  access  to  all  sites,  interviews  with  scientists,  and  landing  and  over 
flight  rights  (ISG,  Volume  I,  2004,  p.  43).  According  to  Chief  U.N.  Weapons  Inspector 
David  Kay: 

Iraq  was  in  clear  violation  of  the  tenns  of  Resolution  1441.  Resolution 
1441  required  that  Iraq  report  all  of  its  activities,  one  last  chance  to  come 
clean  about  what  it  had.  We  have  discovered  hundreds  of  cases,  based  on 
both  documents,  physical  evidence  and  the  testimony  of  Iraqis,  of 
activities  that  were  prohibited  under  the  initial  U.N.  Resolution  687  and 
that  should  have  been  reported  under  1441,  with  Iraqi  testimony  that  not 
only  did  they  not  tell  the  U.N.  about  this,  they  were  instructed  not  to  do  it, 
and  they  hid  material  (Kay,  1995). 

For  many  in  the  Intelligence  Community,  it  seemed  inconceivable  that  Saddam 
would  thumb  his  nose  at  the  U.N.  and  risk  outright  war  with  the  United  States  if  he 
wasn’t  hiding  something.  “Due  in  no  small  part  to  the  pre-war  situational  context,  the 
Intelligence  Community  failed  to  examine  seriously  the  scenario  which  later  turned  out  to 
be  the  case:  domestic  or  regional  political  pressures  or  some  other  factors  prompted 
Saddam  Hussein  to  destroy  his  stockpiles  and  to  forswear  active  development  of  weapons 
of  mass  destruction  after  the  first  Gulf  War”  (Silberman  &  Robb,  2005,  p.  147). 
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VI.  CONCLUSION 


Commenting  on  Pearl  Harbor,  Roberta  Wohlstetter  found  it  easy  after  the  event  to 
sort  the  relevant  from  the  irrelevant  signals.  “After  the  event,  of  course,  a  signal  is 
always  crystal  clear;  we  can  now  see  what  disaster  it  was  signaling  since  the  disaster  has 
occurred.  But  before  the  event  it  is  obscure  and  pregnant  with  conflicting  meanings”  (as 
cited  in  Keane  &  Hamilton,  2004,  p.  339).  This  is  also  quite  clearly  the  case  with  the 
Intelligence  Community’s  pre-war  assessment  of  Iraqi  weapons  of  mass  destruction. 

We  now  know  that  the  Intelligence  Community’s  conclusions  regarding  Iraq’s 
nuclear  weapons  capabilities  were  almost  entirely  unfounded.  Based  on  its  extensive 
post-war  investigations,  the  Iraq  Survey  Group  concluded  that  Iraq  had  not  tried  to 
reconstitute  a  capability  to  produce  nuclear  weapons  after  1991  (ISG  Volume  II,  2004,  p. 
7).  Moreover,  the  ISG  judged  that  Iraq’s  work  on  uranium  enrichment,  including 
development  of  gas  centrifuges,  essentially  ended  in  1991,  and  that  its  ability  to 
reconstitute  its  enrichment  program  progressively  decayed  after  that  time.  In  particular, 
the  ISG  found  that  Iraq’s  effort  to  procure  the  aluminum  tubes  was  “best  explained  by  its 
efforts  to  produce  81 -mm  rockets”  (p.  9);  and  uncovered  no  evidence  that  the  tubes  were 
intended  for  use  in  a  gas  centrifuge  as  the  Intelligence  Community  believed.  Finally,  the 
ISG  found  no  evidence  to  show  that  Iraq  sought  uranium  from  abroad  after  1991  or 
renewed  indigenous  production  of  such  material  (p.  9). 

We  now  know  that  the  Intelligence  Community’s  findings  concerning  Iraq’s 
biological  weapons  programs  were  similarly  without  merit.  In  fact,  the  Iraq  Survey 
Group  concluded  that  by  1992,  Iraq  had  unilaterally  destroyed  its  undeclared  biological 
weapons  stocks  and  probably  destroyed  its  remaining  holdings  of  bulk  BW  agent. 
Moreover,  the  ISG  found  no  direct  evidence  that  Iraq,  after  1996,  had  plans  for  a  new 
BW  program  or  was  conducting  BW-specific  work  for  military  purposes  (ISG  Volume 
III,  2004,  p.  I).21  Further,  although  Iraq  had  retained  some  dual-use  equipment  and 


21  In  fact,  from  the  mid-1990s,  despite  evidence  of  continuing  interest  in  nuclear  and  chemical 
weapons,  the  ISG  found  absolutely  no  discussion  or  even  interest  in  BW  at  the  Presidential  level  (ISG 
Volume  III,  2004,  p.  1). 
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intellectual  capital,  the  ISG  found  no  evidence  of  any  mobile  BW  program.22  Finally,  the 
credibility  of  the  human  source  (code-named  “Curveball”)  on  which  much  of  the  pre-war 
assessment  had  been  based  came  into  question  around  the  time  of  the  publication  of  the 
NIE  and  collapsed  under  scrutiny  in  the  months  following  the  war  (Silberman  &  Robb, 
2005,  p.  80). 

We  now  know  that  the  Intelligence  Community’s  findings  concerning  Iraq’s 
chemical  weapons  programs  were  also  mistaken.  The  Iraq  Survey  Group  concluded  that 
Iraq  had  unilaterally  destroyed  its  undeclared  chemical  weapons  stockpile  in  1991. 
Further,  it  found  no  credible  indications  that  Baghdad  had  resumed  production  of 
chemical  munitions  thereafter,  a  policy  ISG  attributes  to  Baghdad’s  desire  to  see 
sanctions  lifted,  or  rendered  ineffectual,  or  its  fear  of  force  against  it  should  WMD  be 
discovered  (ISG  Volume  III,  2004,  p.  1).  In  fact,  the  ISG  noted  that  the  only  chemical 
weapons  it  recovered  after  the  war  were  those  manufactured  the  first  Gulf  War,  and  that 
after  1991  only  small,  covert  labs  were  maintained  to  research  chemicals  and  poisons, 
primarily  for  intelligence  operations  (ISG  Volume  III,  2004,  p.  3). 23  Finally,  the  ISG 
detennined  that  pre-war  concerns  of  Iraqi  plans  to  use  chemical  weapons  if  Coalition 
forces  crossed  certain  defensive  “red  lines”  were  groundless;  the  “red  lines”  referred  to 
conventional  military  planning  only  (Silberman  &  Robb,  2005,  p.  1 12). 

How  could  the  Intelligence  Community  have  gotten  it  so  wrong?  By  now,  the 
answer  should  be  apparent;  though  a  short  summary  may  be  in  order. 

Lacking  reliable  data  about  the  true  nature  of  Iraq’s  nuclear,  biological,  and 
chemical  weapons  programs,  the  starting  point  of  most  analysts  was  firmly  anchored  in 
Baghdad’s  history:  its  past  use  of  chemical  weapons,  its  successful  concealment  of  WMD 
programs  both  before  and  after  the  Gulf  War,  and  its  failure  to  account  for  previously 

22  Any  attempt  to  create  a  new  BW  program  after  1996  would  have  encountered  a  range  of  major 
hurdles.  The  years  following  Desert  Storm  wrought  a  steady  degradation  of  Iraq’s  industrial  base:  new 
equipment  and  spare  parts  for  existing  machinery  became  difficult  and  expensive  to  obtain,  standards  of 
maintenance  declined,  staff  could  not  receive  training  abroad,  and  foreign  technical  assistance  was  almost 
impossible  to  get.  Additionally,  Iraq’s  infrastructure  and  public  utilities  were  crumbling.  New  large 
projects,  particularly  if  they  required  special  foreign  equipment  and  expertise,  would  attract  international 
attention.  U.N.  monitoring  of  dual-use  facilities  up  to  the  end  of  1998  made  their  use  for  clandestine 
purpose  complicated  hill  of  risk  (ISG  Volume  III,  2004,  p.  2). 

23  ISG  uncovered  information  that  the  Iraqi  Intelligence  Service  (IIS)  maintained  a  set  of  undeclared 
covert  laboratories  to  research  and  test  various  chemicals  and  poisons,  primarily  for  intelligence  operations, 
from  1991  to  2003  (ISG  Volume  III,  2004,  p.  3). 
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declared  stockpiles.  “The  analysts’  operating  hypothesis,  therefore,  was  that  Iraq 
probably  still  possessed  hidden  chemical  and  biological  weapons,  was  still  seeking  to 
rebuild  its  nuclear  weapons  program,  and  was  seeking  to  increase  its  capability  to 
produce  and  deliver  chemical  and  biological  weapons”  (Silbennan  &  Robb,  2005,  p.  49). 
The  WMD  Commission  found  that 

In  essence,  analysts  shifted  the  burden  of  proof,  requiring  evidence  that 
Iraq  did  not  have  WMD.  More  troubling,  some  analysts  started  to 
disregard  evidence  that  did  not  support  their  premise.  Chastened  by  the 
effectiveness  of  Iraq’s  deceptions  before  the  Gulf  War,  they  viewed 
contradictory  information  not  as  evidence  that  their  premise  might  be 
mistaken,  but  as  evidence  that  Iraq  was  continuing  to  conceal  its  weapons 
programs  (Silbennan  &  Robb,  2005,  p.  49). 

Once  analysts  made  a  “first  pass”  at  this  problem,  faulty  indicators  became  much 
more  representative  and  much  more  available  than  they  otherwise  may  have  been. 
Biased  judgments  became  remarkably  resistant  to  further  information,  alternative  modes 
of  reasoning,  and  even  logical  or  evidential  challenges.  Attempts  to  integrate  new 
information  found  many  analysts  and  organizations  surprisingly  conservative,  that  is, 
willing  to  yield  ground  only  grudgingly  and  primed  to  challenge  the  relevance,  reliability, 
or  authority  of  subsequent  information  or  logical  consideration.  As  a  result,  just  as 
Daniel  Kahneman,  Amos  Tversky,  Richard  Nisbett,  Lee  Ross,  and  a  host  of  other 
cognitive  psychologists  might  have  predicted,  the  simple  heuristics  of  representativeness 
and  availability  had  disproportionate  impact  in  the  ultimate  assessment. 

More  than  three  years  after  the  dramatic  failures  of  the  Intelligence  Community 
regarding  Iraqi  weapons  of  mass  destruction,  it  is  easy  to  find  the  mistakes  in  its 
judgments.  Yet  “intelligence  analysis  is  a  tricky  business.  Analysts  are  often  forced  to 
make  predictions  in  the  absence  of  clear  evidence  -  and  then  are  pilloried  after  twenty- 
twenty  hindsight  reveals  that  they  failed  to  paint  a  full  picture  from  conflicting  and 
scattered  pieces  of  evidence”  (Silbennan  &  Robb,  2005,  p.  168).  And  while  we  should 
be  reluctant  to  fonn  too  harsh  a  judgment  about  the  analysts  and  decision-makers  who 
made  them,  we  cannot  ignore  the  fact  that  few  within  the  Intelligence  Community  have 
acknowledged  the  role  of  perceptual  and  cognitive  bias  in  intelligence  analysis  and 
decision-making,  and  even  fewer  have  sought  to  do  anything  about  it. 
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